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ABSTRACT 

The Education Voucher Demonstration is a large-scale 
educational and social intervention that began in the Alum Rock Union 
Elementary School District, San Jose, California, in September 1972, 
First funded by the Office of Economic Opportunity, it is now 
sponsored by the National Institute of Education, This technical 
appendix is the third of four Rand reports that describe and analyze 
the workings of an education voucher demonstration during its first 
year (academic year 1972-73) • The appendixes in this volume contain 
additional technical material related to the contents of the main 
volume of the series and are numbered to correspond with the section 
numbers of that report, (Author) 
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This Technical Appendix is the third of four Rand reports that describe and 
analyze the workings of an education voucher demonstration during its first year 
(academic year 1972-73). 

The other volumes of the series are: 

R-1495-NIE, A Public School Voucher Demonstration: The First Year at Alum 

Rock, Daniel Weiler, Study Director, June 1974 
R-1495/1-NIE, A Public School Voucher Demonstration: The First Year at Alum 

Rock, Summary and Conclusions, Daniel Weiler, Study Director, June 1974 
R-1495/3-NIE, A Public School Voucher Demonstration: The First Year at Alum 

Rock, Documentary Appendix, Daniel Weiler, Study Director, June 1974 

The appendixes in this volume contain additional technical material related to 
the contents of R-1495-NIE, the main volume of the series, and are numbered to 
correspond with the section numbers of that report. All four volumes of the study 
were prepared pursuant to NIE Contract B2C-5326. 

The Education Voucher Demonstration is a large-scale educational and social 
intervention that began in the Alum Rock Union Elementary School District, San 
Jose, California, in September 1972. First funded by the Office of Economic Oppor- 
tunity, it is now sponsored by the National Institute of Education. 

Since April 1972, Rand has been collecting and analyzing data related to the 
demonstration. Our work documents events and outcomes in the demonstration; 
analyzes social, political, economic, and educational impacts of the demonstration; 
and identifies implications of the voucher concept for federal, state, and local educa- 
tion policies. 
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Appendix I 
THE USE OF COMPARISON GROUPS 
by Pierce Barker 



Much of the research in the social sciences deals with change. That is especially 
true in evaluation research and the analysis of social demonstrations, where the 
objective is to assess the consequences oi'some intervention. In the most general 
terms, two issues or questions are involved: (1) Did any change occur? (2) If so, why? 

In most cases, the first question is far easier to answer; measurements are made 
at two points in time, and a decision, usually guided by statistical criteria, is made 
based on differences between the two measurements on dimensions of interest. 

People are usually more interested in ^he second question, however. That is, one 
would like to know whether any observed change can be causally linked to the 
intervention; or, to go to the other extreme, whether the change can be accounted 
for by factors apart from the intervention. That is usually a matter of degree, of 
course; i.e., the question properly is how much of any observed change can reliably 
be attributed to the intervention. 

Such measures obviously involve some sort of comparison process, of which 
Campbell and Stanley (1963) have the following to say: 

Basic to scientific evidence ... is the process of comparison, of recording 
differences, or of contrast. Any appearance of absolute knowledge, or intrin- 
sic knowledge about singulsr isolated objects, is found to be illusory upon 
analysis. Securing scientific evidence involves making at least one compari- 
son. For such a comparison to be useful, both sides of the comparison should 
be made with similar care and precision. 

Even assuming that we have met the criteria set out in this dogmatic prescrip- 
tion, however, we are only part way home. That is, it enables us to verify that a 
change occurred, if it did, but it does nothing to explain why it occurred. At this point 
the idea of the control (or, better comparison) condition enters the picture. The 
clearest case of the use of comparison groups occurs in experimentation; and the 
ideal comparison group is identical to the experimental group in all respects but one, 
that one being the intervention or manipulation whose effect the experimenter 
wishes to assess. 

It follows that the usefulness of any comparison group is a function of its degree 
of pretreatment similarity to the group within which the intervention is made. 
Using the comparison group protects the researcher from having his inferences 
about the effect of the intervention weakened by alternative explanations — what 
Campbell and Stanley call "plausible rival hypotheses." If the comparison and 
experimental groups differ in many respects, any one of which may plausibly ex- 
plain observed differences between them aside from the fact that the intervention 
occurred in one and not the other, then the comparison group does not fulfill its 
intended purpose. 

In general, there are two methods of constructing comparison groups: (1) random 
assignment and (2) matching. The first may be illustrat d by the simplest case in 
which there is only one experimental group and one comparison (here, a nonr 
treatment) group. From the available pool, subjects are assigned to one group or the 
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other by a strictly random process. Random assignment ensures that any difference 
between the two groups on any characteristics will be a chance difference; over all 
possible random assignments of subjects to groups, the expected value of the differ- 
ence on any characteristic equals zero. This holds fbi differences on the dependent 
variable prior to the treatment, but it also holds for any or all other characteristics 
on which the subjects may differ and which may be correlated with the dependent 
variable; this includes unmeasured, unmeasurable, and indeed unknown variables. 
The result h\ that, apart from chance departures, the two groups are identical in all 
respects prior to the treatment or intervention. The purpose of statistical tests of 
significance is, of course, to allow us to assess the likelihood that any results we get 
are the result of chance pretreatment differences. 

Matching in isolation ( i.e., without concurrent randomization) should only occur 
in circumstances precluding random assignment. The purpose of matching is to 
deliberately equate the two groups on variables thought to be relevant to the out- 
come measure at hand, so that preexisting differences on these matching variables 
cannot be invoked as a rival to the inference that the treatment had an effect. 
Matching without randomization is in general not recommended, if only because in 
matching on one set of characteristics, one may easily mismatch on other, perhaps 
more crucial, dimensions. 

It is in theory possible to establish multiple comparison groups, some of which 
are matched with the experimental group on certain characteristics, some matched 
on others, and this process may be extended as far as one wishes. Then, if one finds 
diffe^^ences between the experimental group and the control groups, a plausible rival 
hypothesis for one comparison may be ruled out by matching on another. 

In large-scale social interventions, randomization is oflen impossible. On the 
other hand, one may nevertheless perform valid m/er;ia/ comparisons, given certain 
conditions; that is, one may seek to compare the results of an overall intervention 
on subgroups of interest within the total group receiving the intervention. For 
example, if the overall intervention is realized within the site in different ways, one 
may in theory compare the results of these diverse realizations. The principal prob- 
lem here usually results from the fact that individuals getting the different 'treat- 
ments' are self-selected; that is, they choose one treatment or another for reasons 
presumably relevant to the treatments. Take as an example students who choose to 
participate in a remedial reading program. Given random assignment, one would 
expect that there would be no significant difference between treatments on initial 
reading skills. Given self-selection, one would of course expect initial differences: 
poorer readers are probably more likely to sign up for remedial programs. Moreover, 
one might well expect to find differing rates of improvement between these groups. 

In this situation, where creatment-pretest correlations are not (expected to be 
zero, several strategies exist. One of these would seek to explain the correlation in 
terms of known group differences; given that the residual correlation approximated 
zero, we might then be able to make comparisons similar to those in a randomized 
experiment. A related strategy would examine changes in the size of these correla- 
tions over time. Still another might examine differences in regression functions 
within groups. (See, e.g., Kenny, Cronbach and Furby, 1970.) 

Still, the impossibility of random assignment to treatments does not absolutely 
cripple us. One recourse, apart from the internal comparisons already discussed, is 
to select norjequivalent groups (in the sense of equivalence implicit in the idea of 
random assignment). As we have already indicated, establishing and measuring 
enoughsxxch groups will in theory enable us to make inferences as strong as we wish. 
The key question, however, is of course the degree and kind of lack of equivalence. 
Note that, we are now dealing, not with internal comparisons, i.e., of groups each of 
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which received some variety of intervention, but with comparisons of a ''project" 
Kroiip with some other "nonproject" group. In the present context, we would com- 
pare the "voucher" with a ''nonvoucher" site. 

Another variant of this type of comparison involves a treatment group and a 
"reasonably" nonequivalent group. Assume that, on dimensions of interest, we could 
secure measurements at a number of time points prior to the introduction of the 
"treatment" to one of the groups, and could moreover continue these parallel meas- 
urements for a number of time points subsequent to the application of the ''treat- 
ment." Then we could look for departures of the response curves from their former 
trend lines, in an effort to answer at least two questions: (1) are the departures from 
the previous trends real (i.e., usually, statistically significant); and (2) are they 
reasonably attributable to the intervention? The simplest case, of course, would be 
a marked departure in the "treatment" group and a continuation of the former 
trend in the comparison group. 

Clearly, the preceding discussion has done no more than sketch the varieties of 
alternative approaches to introducing comparison groups into research. The prob- 
lem here, as always, is to apply the theory in a maximally beneficial way within the 
circumstantial constraints of a real world problem. Specifically, what are we to say 
about the voucher demonstration in Alum Rock? 

It should be clear that random assignment to the voucher "treatment" at the 
district level is not worth considering.* This is even more true of what we might 
consider "treatments" — for example, educational programs with differing emphases 
— within the district's voucher schools. The basic idea of the voucher after all, is to 
provide students and their parents with maximally free choice of educational alter- 
natives; the very idea of random assignment is antithetical to this orientation. 

Alum Rock is in many respects a special site. If we consider the average SES of 
the local population, the unusually high degree of migration, and the ethnic mix, 
it is clear that it would be difficult to locate a "reasonably" nonequivalent compari- 
son district. Assuming that such a district (or, better, districts) could be found, the 
problems of securing the cooperation of these other districts for what must inevita- 
bly be a considerable degree of interruption of their routines, without palpable 
advantage to them, and of mounting and financing parallel assessment efforts, are 
formidable. 

On the other hand, not all of the schools in Alum Rock chose to participate in 
the voucher demonstration, so that withinihe school district, there are voucher and 
nonvoucher schools. On the reasonable assumption that the district is itself more 
likely to be homogeneous on most relevant variables than this district would be with 
other districts that might be selected for comparisons, the nonvoucher schools, while 
not a random sample of district schools, would seem prima facie reasonable candi- 
dates for the status of comparison group. 

From the formal viewpoint, it is true that within-district comparisons between 

' Perha[)S it is well to mention that the reasons are not logical, but contingent, which means the 
situation could change. For example, if there had been widespread demand among school districts for a 
voucher demonstration — if there had been a seller's market — something like random assignment of 
districts would have been more than possible. In reality, the reverse was true — not an unusual situation 
when a controversial educational innovation is offered, which calls for major internal changes in the 
district's system. (See Weiss, 1974.) 

ff the demand (and financing) were present, we would in fact recommend something like a 2" factorial 
experimental design with site sampling as the next or near-next step in exploring the voucher idea. On 
the r)ther hand, it would have been foolhardy to recommend anything nearly so elaborate as a first step, 
without so much as a test under field conditions: there was no operational evidence that anything like 
the voucher system could be successfully installed in a field site. Without this minimal knowledg*^, which 
could be gained only by installing something like a voucher system and watching what happened, we 
would have been attempting to walk before we learned to crawl. 



the voucher and the nonvoucher schools would probably be rather closely matched 
on just such variables as were noted above: SES, migration rate, ethnic mix, and so 
on. However, the fact is that nonvoucher schools in Alum Rock had that status 
precisely because they did not want to be part of the voucher u/^onstration; a 
fortiori, they were unwilling to tolerate the study and analysis activities that the 
voucher schools barely tolerate as part of the demonstration. 

It is also worth noting that the very fact of nonparticipation suggests differences 
from the voucher schools which, if not explicitly known, are almost certainly rele- 
vant to comparisons that might be made. In fact, it is just such suspected and not 
explicitly identifiable differences that can easily weaken if not invalidate inferences 
about treatment effects. Suppose, for example, that voucher and nonvoucher schools 
did differ considerably on the factors of staff morale and cohesion, and degree and 
style of administrative leadership (a supposition that, in passing, is not unlikely). 
Strong impressions that such differences, even if not reliably measurable, were 
nonetheless real and influential, might easil]/ lead to the conclusion that these 
differences offered a potent rival hypothesis to the proposition that response differ- 
ences were attributable to the presence of the voucher demonstration. (Note below, 
however, that an alternative strategy of some potential usefulness may be based 
upon the fart that new schools are entering the demonstration in the second year.) 

Moreover, as mentioned above, the strong likelihood is that not one, but multiple 
comparisons would be necessary, in view of the very individual nature of the Alum 
Rock district, not only to strengthen inferences in this demonstration, but to im- 
prove the possibilities for generalization from this study. It is the voucher concept 
itseF. and thus a district level of analysis, that is of paramount importance here. 

However, this should not obscure the value of internal comparisons, which can 
of course be made. The Alum Rock voucher schools are not, afler all, completely 
homogeneous; differences in educational alternatives have been planned and imple- 
mented within the demonstration site. Schools do differ in leadership style, and the 
impact of the voucher presence itself upon this factor is of considerable interest. The 
notion that there exist unassessed treatment-ability (or aptitude or personality) 
interactions — that, for example, the effects of different teaching methods or cur- 
ricula may be specific to certain types of students — is seen by some as a likely 
explanation of the common finding that educational innovations have little or no 
overall impact (e.g., Anderson, 1970). It maybe possible to explore this idea in Alum 
Rock data. 

This is not intended to be a brief against comparison and control in large-cjcale 
social intervention research. The entry into the voucher demonstration in its second 
year of Alum Rock schools which did not participate in the first year affords oppor- 
tunities for comparison in what we may call a lead-lag design. That is, it should be 
possible to look at effects of the demonstration over time by comparing second- with 
first-year schools on the same dimensions. Note that this does not obviate the neces- 
sity of identifying and controlling differences between these two classes of schools 
on other variables; but, by the same logic that applied to inferences between voucher 
and nonvoucher schools within the district, the likelihood is that differences be- 
tween first- and second-year schools are less than those between present voucher and 
nonvoucher schools. 

The introduction of suitable comparison groups into future research on the 
voucher concept is much to be desired, but entails serious considerations beyond the 
mere decision to proceed with it. For one thing, ample lead time is necessary, so that 
"suitable" groups may be selected (in the sense of the criteria sketched above); and 
prior to the choice, of course, sufficient measurement must be done to enable their 
suitability to be assessed. For another, the cost of additional groups will clearly be 



greater than thai for one only; what is involved is in fact something near a duplica- 
Hon of assessment in each such group. Again, this is not to say that it cannot and 
should not be done, only that there are costs and prerequisites which must be clearly 
stated and met. It is characteristic of projects of this kind that their period of 
gestation is relatively long, but once they come to term, they start up almost over- 
night.^ It is hardly necessary to point out that this trend does not promise much by 
vvav of lead time. 

In many ways, the Alum Rock experience has been very useful for its testimony 
on the question of comparison groups. For one thing, it is providing a good deal of 
acute appreciation of the logistics involved in assessment methodology, which will 
enable us to undertake similar projects on an expanded scale, including the estab- 
lishment of relevant comparisons, without the necessityof repeating the experiental 
learning that is inevitably involved in undertakings of this magnitude. 

The question, then, is not one of the usefulness of comparison groups in the 
abstract, but of their availability and relevance in the current context. Given the 
circumstances of the establishment of the present demonstration, a clearcut com- 
parison group was not a viable possibility. This may well not be true of future 
projects, but we are now concerned with Alum Rock. Inclusion of relevant groups 
would inevitably have delayed the establishment of the Alum Rock demonstration, 
and at the time this was not feasible. 

However, it has been one purpose of this brief review to indicate that the absence 
of formal extramural comparison groups is not necessarily a fatal disability; nor 
would their inclusion have been a blanket assurance that they would have been 
useful. The particular circumstances that form the context of a specific study are of 
paramount importance, not the researcher's professional dogma. 



The final ^o-ahead for Alum Rock came in April 1972; the demonstration got under way in Septem- 
ber, after a summer of intensive preparation. For a project of this size and complexity, this is tantamount 
to no lead time at all; but if a second site is selected, the pattern seems likely to be repeated. 



REFERENCES 

Anderson, G. J., ^'Effects o'Classroom Social Climate on Individual Learning," Am. 
Ed. Res. J.. Vol. 7, 1970, pp. 135-151. 

Campbell, D. T., and J. C. Stanley, "Experimental and Quasi-Experimental Designs 
for Research on Teaching," in N. L. Gage (ed.), Handbook of Research on 
Teaching, Rand McNally and Co., Chicago, 1963. 

Cronbach, L. J., and L. Furoy, "How We Should Measure ^Change'^r Should We?" 
Psychological Bulletin, 1970, Vol. 74, No. 1, pp. 68-80. 

Kenny, D. A., "Temporal Changes in Treatment-Effect Correlations: A Quasi-Experi- 
mental Model for Panel Studies" (in press). 

Weiss, E. H., "The Missing Variable in Input-Output Studies," Planning and 
Change. Vol. 4, 1974, pp. 203-210. 



Appendix III 



THE FEDERAL GRANT FOR THE VOUCHER 
DEMONSTRATION 

by Sue A. Haggart 



Alum Rock's 1972-73 income from federal sources, both direct federal funds and 
federal funds channeled through the state machinery, almost doubled over the 
previous year (see Table IIM). The voucher-related funds shifled from less than 0.5 
percent of total 1971-72 funds from federal sources to about 50 percent for 1972-73. 
State and local income also increased, but made up about the same percentages of 
district income. Basic equalization aid from the state, however, increased from $6.3 
to $7.0 million but decreased as a percentage of total income. The total 1972-73 
voucher demonstration funds represent about 9 percent of the adjusted district 
income of $17,91C 624. Another way of assessing the impact of the voucher demon- 
stration is to look at the budget's Current Expenses Total. This total excludes 
Transportation, Food Services, Community Services, and Capital Outlay; for 1972-73 
it was $15,672,282. The voucher funding of $1,585,756 is approximately 10 percent 
of Total Current Expenses. 

The percentage distribution of funds by source for the voucher schools and the 
nonvoucher schools provides another picture of the impact of the voucher grant 
(Table III-2). On the average, the compensatory voucher funds accounted for 25.9 
percent of the income of voucher schools. For the Title I schools, the average income 
from Title I was 32.9 percent. 



NATURE OF THE FEDERAL GRANT 

The allocation and earmarking of funds for the elementary education voucher 
demonstration in Alum Rock resulted from bargaining between local district offi- 
cials and officials from OEO, and later from NIE. A summary of funding as a result 
of agreements to date is shown in Table III-3 in the format prescribed by the Office 
of Management and Budget. Details by cost category are shown in Table III-4. Most 
of the increases took place in Cost Categories 1.1 Salaries and WagesA,3 Consultants 
and Professional Services, and 2.6 Other Direct Costs. Agreements of a substantive 
nature were spelled out in subcategories of these broad categories and were stated, 
in some cases, as special conditions governing the use of funds. These increases were 
covered in two negotiated additional grants. The data in Tables III-3 and III-4 show 
the combined funding for the education voucher demonstration through June 30, 
1975 as of March 14, 1973. 

A breakdown of the 1972-73 and 1973-74 budgets, shown in Table III-5, provides 
information about the activities in support of the voucher demonstration. (The 
expenditure categories for 1973-74 were developed by the Sequoia Institute, and the 
1972-73 grant and the prior grant funds were allocated to the same expenditure 
categories.) A description of each category provides insights into the nature of the 
federal grant. A comparison of the first and second year grants helps to understand 
the operation of the voucher demonstration. 
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Table lll-l 

ALUM ROCK UNION KLKMKNTAHY SCHOOL DISTRICT: FINAL INCOME BUDCET, 
19 71- 72 , AND PKO.JKCTFJ) INCOME BUDCET. l972-*73 



1 ncome "ourc e 


1971-72 
Funds 


Percent 

of 
Budget 


1972-73 
Funds'^ 


Percent 

of 
Budget 


NuL balance 


$ 386, 90A 


2,7 


251,592 


1, 5 


Fedora I 
PL 8 7A 

PL «S A52 OEO 
Title I ESE.\ 
Title I Migrant 
Title 11 
Title TIT MDI-:A 

rit lo vn h'SEA 

lUID Model Cities 
Calif. Criminal .Listice 
Prcsi'hool. Ed, Aid 


60,000 
52,765 

885,517 
28,605 
27,767 
2,671 

10A,22A 

98,230 
1,259,779 


8,8 


70,000 
933,170b 
802,202 

40,000 
3,943 
119,596 
155,000 
87,532 
142, 582 
2,35^. ,025 


13,6 


State 

Basic Equal, Aid 


6, 334,260 


44, 3 


7,024,182 


40, 7 


Ediic , hand ii."appod 
Mentally i\if tod 
Ment<»lly retarded 
Physically handicapped 
Sev, mentally retarded 


159,500 
13,460 

100,618 

289,900 
47,477 
27 230 

638 [ 185 


4,5 


278,195 
27, 740 
98,520 

309,515 
45,420 
43 000 

"802,390 


4,6 


Allow, transportation 

El em, school reading 

County income 

Spfcial Pfivsical Education 


111,946 
96,530 
7.427 




103,000 
96,530 
12,000 
11,625 






215,903 


1.5 


223,155 


1,3 


Local 

District trixes 
Area wide taxes 
Other local 
Otlier tuition 


3,900,381 
1.505.918 
43,158 
18,000 
5,467,457 


38, 2 


4,658,143 
1,884,415 
143,902 
12.000 
6,602,694 


38, 3 


Total 


$14,302,488 


100.0 


$17,258,038 


100,0 



SOURCE: The Final Income anf' Expenditure Budget of the Alum Rock 
Union Elementary School District, 



^^Projected income as of July 31, 1972. 

^Does not include total voucher demonstration grant. The 1972-73 
combined grant was $1,585,756* 



Table 111-2 



Bl'DCKTKD KXPKNUTTUKKS PKK RKCIII.AK STUDKN'T AND SDUKCK OK rUNDS l-OK 
VOUCHER SCHOOLS AND NO.WoDCIIKK SCHOOLS, 19/2-73 













s'ourcu of l'\iuc 














(Pi?rcenta^u Distr ibut ion) 






Ml im K » » T" n f 

tlUlll w L. L L' L 




LjSL ITTIU L 




Model 


Coinp . 






S '1 

. L lOO 


CI r 1 1 f-l »n n t 1^^^ 




B I k1 ^ 0 1 $ 


111/141^^ 


Cit ies 


Vouclier 


Title I 


fit- hii r 




5 36 


816 


4 37 3 76 


65.9 






30. 2 


X Q 


CasseJ 1^ 


671 


704 


472 384 


74.3 


1 .9 


23.8 






Conlf f 


372 


869 


323 268 


63.0 








3 7 . 0^ 


Cur p ^nn 


487 


552 


268 ,8 24 


99.4 


0.6 










634 


563 


356 942 


95,5 


4.5 








Goss*" 


708 


682 


482 ,856 


67.2 


2.8 


26,2 




3 . 8 


Htibbu rd 


522 


782 


408, 204 


68.3 


(e) 




31.7 




I 1 nilA V f "^ra 


702 


544 


381 888 


96. 3 








3 . 7 


L V nd 3 1 6 


612 


4 52 


276 624 


99.4 


0.6 


11 






Mayfalr 


684 


724 


495,216 


66.4 







33. 6 


— 


McColLam^ 


584 


669 


390,696 


70.5 


0.4 


29.1 








781 


662 


517 ,022 


68.8 


3.9 


24.0 


_ ^ 


3 . \ 


Miller^ 


4 58 


898 


411 ,284 


70.9 


2,8 


26.3 


— 




Mt Ham^ 1 tan 


8 


1441 


11 , 520 


100. 0 










Painter 


295 


654 


192 ,930 


99.2 


0.8^ 








Rogers 


582 


503 


292,746 


99,5 


0.5 








Ryan 


661 


552 


364,872 


100,0 










San Antonio 


355 


915 


324,825 


68,1 






31.9 




Shields 


641 


463 


296,783 


99,5 


0.5 








Slonaker 


591 


722 


426,702 


62.2 


(e) 




33.3 


4.5 


Fisher 


1046 


665 


695, 590 


94,6 


3,1 






2.3 


George 


631 


670 


422,770 


96.7 








3.3 


Mathson 


651 


893 


581,343 


91.8 


2.6 






5.6 


Pala^ 


601 


B59 


516,259 


79.4 




17,8 




2.8 


Sheppard 


991 


716 


709,556 


96.1 


1.8 






2.1 



^Special education students and funds are excluded. Enrollment figures and distribution 
data are from unpublished district sources. 



The elementary school program is 111 and the middle school program is 141, with added 
programs 142-151 for middle subjects such as art, math, science, industrial arts, etc. 

Q 

1972-73 voucher schools. 
^Title VII, Bilingual Education. 
^Less than 0, 5. 



Table III-3 



VOUCHER DEMONSTRATION BUDGET SUMMARY, 1971-75 



Cost Category 


1971-72 


1972-73 


1973-74 


1974-75 


Total 


1 


1 


Salaries and wages 


10,132 


188,750 


271,863 


266, 387 


737,132 


1 


2 


Fringe benefits 


2,7V4 


31,500 


49,822 


47,950 


132,066 


1 


3 


Consultant and 














professional services 


12,223 


218,190 


286,962 


190,148 


707,523 


2 


1 


Travel 


412 


7,000 


15,000 


15,000 


37,412 


2 


2 


Space costs and rentals 




9,200 


54,400 


54,400 


118.000 


2 


3 


Consumable supplies 


759 


5,000 


10.000 


10,500 


26,259 


2 


4 


Lease and purchase of 
















equipment 


1,642 


30,000 


20,500 


21,000 


73,142 


2 


5 


Investment capital 












2 


6 


Other direct costs 


2,418 


1,039,153 


2,111,450 


1,679,450 


4,832,471 


3 


0 


Indirect costs 


3,498 


56,963 


104,871 


85,851 


251,183 






Year total 


33,878 


1,585,756 


2,924,868 


2,370,686 


6,915,188 






Cumulative total 


33,878 


J ,619,634 


4,544,502 


6,915,188 


6,915,188 



SOUI^CE: Federal grant data as of March 197 3. 

NOTE: Dates are /^pril 1 through June 30 for each column. 



* 



Table [I 1-4 

Al.l'M ROCK Vo'.'(;ilKR HI : K i' : CoMB I Nl'l) KCNDINi: AS oK MARCH i'H \ 



r tt-m 



KV 71-72 



!'V 7J-7.J 



FV 74-7-> 



Total 



1.1 Salarlus and Wa>;t's 

• F u 1 1 - T Ime , P u rtn a no n t 
t\ccou\\iant 
2-Coiniselors 

2- Secrctar IfS 

3- Clerk-Typlsts 
Attendance Clerk 
Payroll Clerk 
Administrative Assistant 
Messenger /Warehouse 

1- Computer Specialist 

B. Part-Time, Permanen t 
Paracounselors 

2- Researrh Assistants 
Special Education Counselor 

C. Temporary 
Translator 
Substitute Salaries 
Secretaries for Attendance 
Evaluation Survey Personnel 

Subtotal 1.1 

1.2 Fringe Benefits 

Annuity Fund 3.2% Certified 
Public Employment Retirement 7.25% 
OASDHT 5.2% Classified 
Employment Insurance Teacher 

$427.80 Certified 
Employment Insurance Other 

$427.80 Classified 
Unemployment Insurance 

3.4% of $42,000 
Workmans Compensation Insurance 

0.006% Certified and Classified 

Subtotal 1.2 (18% of 1.1) 

1.3 Consultants and Professional 

Services 

A. Voucher Project Management 

B. Staff and School Training 

C. Attendance and Fiscal Accounting 

D. Independent Audit 

E. Systems Development 

Subtotal 1.3 

2.1 Travel 

Subtotal 2.1 

2.2 Space Cost and Rental 



$ 1,025 
2,215 
558 



6,334 



$10,132 



$ 2,794 



$ 4,217 
8,006 



$12,223 



$ 412 



A. Voucher Staff 

B. Student Space Voucher 

Subtotal 2.2 



10,332 
33,405 
13,700 
6,036 
4,225 



2,200 
2,500 



47,452 
4,000 
6,000 



5,000 
50,400 
3,500 



$ 188,750 



31,500 



107,940 
69,000 
5,000 
1,250 
35,000 

218,190 



7,000 



9,200 



9,200 



11,849 
35,034 
15,500 
18,838 
5,355 
8,000 
10,000 
8,500 
10,000 



74,287 
20,000 
6,000 



5,000 
31,800 

11,700 

271,863 



49.822 



$ 134,712 
100,000 
22,000 
4,2 50 
15,000 

$ 286,962 



$ 15,000 



18,400 
36,000 

54 ,4 00 



1 2 , 500 
38,500 
16,000 
19,000 
7,000 
8,400 
10,000 
8,500 
10,000 



80,802 
21,000 



5,000 
17,400 

12,285 

266,387 



47,950 



$ 156,148 

22,000 
7,000 
5,000 

$ 190,148 



$ IS, 000 



18,400 
36,0u0 

54 ,400 



35,706 
109,154 
45,758^ 
43,874 
16,580 
16,400 
20,000 
19,200 
22,500 



208,875 
45,000 
12,000 



15,000 
99,600 
3,500 
23,985 

$ 737,132 



$ 132,066 



$ 414,017 
177,006 
49,000 
12,500 
55,000 

$ 707,523 



$ 37,412 



$ 46,000 
72,000 

$ 118,000 



ERLC 
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Table 1 11-4 (continued) 







V I 


7» 1 \ 




FY 74-75 


Tot. 'I I 


2 . 3 


ConsuTTidblc Siippllt?s 
















Ct 1 K »- t- 1 > 1 
DUDCOlQ L il , J 




e 
•? 




p iU , UUU 


> 10 , 500 


$ 26,2 59 


2 4 


Lease and Purchase of Kqulpnient 
















duD CO Ld i ^ . M 


5 J. , 64 2 


c 
•? 


30 , ()00 


p 20 , jOO 


5? 21 , 000 


$ 73,142 


2 . 6 


Other Direct Cost 








J 




$ 2,418 


A. 


EVAC 


— 




3,000 


5,000 


5,000 


13,000 


B. 


Print Ing 


— 




15,000 


35,000 


35,000 


, 85,000 


(' 


S t ud ent & Finji nc i a 1 Ac c t . 


— 




128,500 


373,000 


200,000 


703,500 


D. 


Incremental Bussing Cost 






11,000 


24,000 


25,000 


60,000 


E . 


Cont inji^ency 








1 4 , 000 


14 , 000 


28 , 000 


F. 


Inservice Training Cost 


— 




280,000 


170,000 


50,000 


500,000 




Compensatory VoLichc*r 






509,100 


1 , /Oil f M }K) 


i , 2o2 , 4 jO 


J ^ 0 /4 , 000 


H. 


Teacher Salary Guarantee 


— 




36,000 


14,000 




50,000 


I . 


Central Service Stai. f Salary 
















Guarantee 


— 




30,000 


50,000 


30,000 


110,000 


J. 


Grants to New Schools 






15,000 


15, 000 




30,000 


K. 


Internal Evaluation 


— 




4,000 


18,000 


18,000 


4 0,000 


L . 


Community Survtjy 






7.553 






7,553 


M. 


School -Communicy Worksliop EVAC 








14.000 




14,000 


N. 


New Voucher School Development 








60,000 




60,000 


0. 


Professional Development Centtr 








35,000 


20,000 


55,000 




Subtotal 2.6 


$ 2,418 


^;i, 039. 153 


$2,111,450 


$1,679,450 


$4,832,471 




Total of above categories 


$30,380 


$1,528,793 


$2,819,997 


$2,284,835 


$6,664 ,005 


3.0 


Indirect Cost (3.72%) 


$ 3.498 


$ 


56,963 


$ 104,871 


$ 85,851 


$ 251,183 




Grand total 


$33,878 


$1,585,756 


$2,924,868 


$2,370,686 


$6,915, 188 


16, 


SOURCE: Letter (including budget) t^ 
1973. 


Superintendent 


Jeff erds 


from Denis 


Doyle of OEO, March 



Table III-5 

(:C)>fPARTSON OK EXPENDITUHE CATEGORIES, 
FIRST ANU SECOMI) YEAR flRANTS 





Prior 


First Year 


Second Year 




Grants , 


Grants , 


Grant s , 


Expenditure Category 


1971-72 


1972-73 


1973-74 


District administrative costs 


4,001 


191,683 


289,004 


Parent counseling staff 


8,549 


91,857 


180,321 


Evaluat ion 




8,000 


49,700 


Fringe benefits 


2,794 


31,500 


49,822 


Inservice training and management 


12,223 


399,400 


336,800 


Travel 


412 


7,000 


15,000 


Space cost, staff 




9,200 


18,400 


Consumable supplies 


759 


5,000 


10,000 


Lease and purchase of equipment 


1,642 


30,000 


20,500 


EVAC and parent involvement 




10,553 


1 9,000 


Printing 




15,000 


35,000 


Buss ing 




11,000 


24,000 


Compensatory voucher 




509,100 


1,282,450 


Student and f inane ial account ing 




128,500 


375,000 


Reserve funds 




81,000 


79,000 


Student space voucher 






36,000 


Indirect costs (3.72%) 


3,498 


56,963 


104,871 


Tutal 


33,878 


1,585,756 


2,924,868 



SOURCE: Federal grant data, March 1973. Sequoia Institute break- 
down of the federal grant for 1973-74 Is used for the 1972-73 and 
1971-72 grants. 
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The budgeted amounts in the categories Tmr^'/and Consumable Supplies sup- 
port the activities of the voucher stafl'and the central district administrative staff 
involved with the voucher demonstration. The category of Lease and Purchase of 
Equipment includes first year funds of $15,000 for a one-time expenditure to provide 
a pool of trailer classroom space. The remainder of this category provides office 
equipment, furniture, and leased trailer space for the voucher staff. The Space Costs 
and Rental covers the space the voucher staff rented from the district at a cost of 
$9,200 fbr the first year and $18,400 (to house a larger staff) for the second year. 

The Student Space Voucfiert'und was set up for the second year to provide a pool 
that could be tapped by schools in need of additional seating capacity. It could not 
be used by schools with a stable or declining enrollment. It serves as a space market 
mechanism. The Student Space Voucher of ^36,000 fbr 1973-74 is calculated at $4 
per student times the 9,000 students in the demonstration. This space voucher was 
added hy the federal agency afler the one-time provision fbr trailer classroom space, 
which, in turn, had been the response to the district's inability to assure an excess 
capacity of 10 percent in seating space in the voucher schools. 

District Administrative Costs, Parent Counseling Staff', EVAC and Parent In- 
volvement cover salaries and wages fbr these activities. Fringe Benefits are tied to 
salaries and wages of the staff involved. The Indirect Costs category includes the 
budgeted amount for the portion of time the central district staff spent on voucher 
demonstration activities. Evaluation covers the operational costs of the internal 
evaluation of the voucher demonstration. 

Printingcowers the cost of printing the informational material about the vouch- 
er demonstration and its mini-school programs for distribution to the parents. In- 
creases are attributable to the increased size of the demonstration. Bussingis in the 
nature of a reserve fund to cover anticipated expenses in transporting students who 
enroll in mini-schools outside their regular attendance area. The increase is also 
attributable to the expanded size of the demonstration. 

Inservice Training and Management cowers the school and staff training con- 
ducted by HRC (Center for Human Resources and Organizational Development), 
substitute salaries fbr teacher release time, and the activities of the Professional 
Development Center in 1973-74. HRC provides management training for the staff 
of* both the central administrative office and the school site. In the first grant, 
$69,000 was budgeted; in the second year, $100,000. (HRC records do not permit the 
allocation of these amounts to school site activities as opposed to central administra- 
tive staff activities. For this reason the HRC-budgeted amounts are maintained as 
Si separate line item in the later analysis of the school and administrative costs of 
':he voucher demonstration.) 

The initial grant provided $30,000 to each voucher school for inservice training 
of teachers. These funds were used almost exclusively to pay teachers for planning 
work undertaken during the summer or at other times when teachers would not be 
working on their regular assignments. A March supplement provided planning 
money for schools entering the demonstration during the second year. Salaries for 
substitute teachers were provided for the six voucher schools in a December 1972 
supplemental grant and for the new schools entering the demonstration in a March 
1973 supplemental grant. 

The Professional Development Center budget of $35,000 was provided for the 
second year of the demonstration to support a new program initiated by the superin- 
tendent. The Center, with a committee of teachers as part of its management, will 
assist teachers in identifying and sharing innovative instructional ideas. Some of the 
substitute salaries will be used to provide release time for teachers in the demonstra- 
tion. 
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Compensatory Vowc/2er represents the amount budgeted to provide a compensa- 
tory voucher for each student eligible for the free lunch program. Its value in the 
first year was $237.77 for elementary and $301.55 for middle level students. The 
1972-73 compensatory voucher fund total was $509,100 ($442,000 provided in the 
initial grant and $67,100 in a supplemental grant). (The $67,100 grant action was 
first carried as $67,010 but was changed to $67,100 in later budget documentation.' 
The value of the compensatory vouchei' in the second year of the demonstration is 
a flat $275 for all eligible students regardless of grade level Those students also 
receiving allocations from ESEA Title I are to receive a "discounted" compensatory 
voucher of $90. The compensatory voucher fund for the second year is $1,282,450. 

The monies budgeted for Student and Financial Accounting cover the expenses 
associated with school-site administration and the operation of systems to track and 
account for the flow of students and dollars within the mini-schools of the voucher 
demonstration. The actual development of the student attendance system and the 
income/outgo budgeting and accounting system was covered in the staffing costs of 
district administration and voucher administration. 

The expenditure category Reserve Funds has three separate funding arrange- 
ments embodied in its rather uninformative title. One is fairly simple; it sets aside 
$15,000 for each of the first two years of the voucher demonstration to assist in the 
development of community-initiated school alternatives. The title of this fund is 
^'Grants to New Schools." Monies in this fund are not to be spent by existing schools 
in the Alum Rock School District that are potential entrants into the voucher 
demonstration. (The development of programs in the Alum Rock public schools was 
considered as a general voucher demonstration activity. The second-year grant 
provided a total of $60,000 in separate grants of $10,000 for each school wishing to 
enter the demonstration in September 1974.) 

The other two funding arrangements under Reserve Funds involve salary gua- 
rantees. The Teacher Salary Guarantee fund is $36,000 for 1972-73 and $14,000 for 
the second year of the demonstration. The district's April 1972 proposal to GEO 
promised full protection of tenure and seniority for participating certified em- 
ployees. Detailed procedures were established to cover the possible displacement of 
a teacher from a mini-school with a declining enrollment. In the first year this fund 
was not needed; the funds can be carried over to the second year. The 1973-74 
increment of $14,000 due to the increased size of the demonstration, along with the 
carryover of $36,000, provides a reserve fund of $50,000 for the teacher salary 
guarantee. 

The third (and final) funding arrangement under Reserve Funds involves a 
salary guarantee for central district staffwho provide central support services to the 
individual schools of Alum Rock. The contingency fund for Central Service Staff 
Salary Guarantee was set up essentially to protect the central district against a 
marked decrease in the demand for central services. The fund size in the 1972-73 
grant was $30,000 (about two-thirds of which was used); in 1973-74 it was increased 
to $50,000 "to offset the greater possibility of salary loss and/or job dislocation" in 
the larger demonstration. 

This mechanism merits additional discussion. At the suggestion of OEO, the 
district sought to identify central services that could be purchased at the discretion 
of the individual schools or mini-schools. The district agreed to the 'Voucherizing" 
of three central services: psychological services, curriculum coordination, and audi- 
ovisual services. In voucherizing a service, the total cost of providing the service is 
divided by the total number of students to obtain a voucherized factor, which is then 
used in allocating the cost of that service or in buying it. Curriculum coordination, 
however, was not voucherized. For the second year, inservice training, which had 
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been a lump sum for each voucher school, was voucherized on a per student basis 
($50 for the first year and $10 for the second year of a schooPs participation). Schools 
and mini-schools then could have the option of buying these services as needed. But 
the district wanted insurance against the schools not buying from the central office. 

Under the voucherizing of these services, an amount equivalent to the dollars 
used to support the furnishing of these three central services to each school would 
become part of the basic voucher income allocated to the school. This school-site 
income, in effect, became discretionary income for the school; these services could 
be purchased from the district or from an outside supplier — all or part. If the schools 
and mini-schools opted not to buy these central services, the district could lose 
money by not having a market for the services it was prepared to provide. Based on 
past experience, it was estimated that the six voucher schools could be provided with 
the three services for a cost of $70,000. It was agreed that the federal agency would 
set up a contingency fund to offset a maximum of $30,000 of this potential loss. (In 
actual practice, the superintendent required that each school commit itself to the 
purchase of an amount of outside services that would not exceed $30,000,) 



RESTRICTIONS, SPECIAL CONDITIONS, AND 
SUPPLEMENTAL GRANTS 

Many of the restrictions and special conditions were related to federal control 
and monitoring of the voucher demonstration monies; those of interest here dealt 
with specific provisions of the demonstration, such as the development of a per pupil 
accounting system and school-level income/outgo budgeting. Supplemental grants 
covered both new activities not included in the initial grant and the increased cost 
of planned activities. 

Restrictions and Special Conditions 

Restrictions were placed on the total funds that could be spent during the period 
prior to June 30, 1973. As of December 2, 1973 only $1,299,407 of the combined grant 
and supplemental grants could be used; on March 14, 1973 the restriction was that 
only $1,585,756 could be used through June 30. Unused funds from prior fiscal years 
and prior grants could be used for the purposes of furthering the grant and the 
voucher experiment. Unused contingency funds for Compensatory Vouchers, Teach- 
er Salary Guarantee, Central Staff Salary Guarantee and Grants to New Schools 
remain available but only within these categories. 

A continuing condition of the grant stated that "a complete per pupil budgeting 
system and operation which contains mini-school income based upon pupil attend- 
ance, and the reporting of complete income/outgo budgets at mini-school level, must 
be developed for and used in the 1973-1974 academic year,*' Additional funds were 
provided to obtain technical assistance for its development. The income/outgo budg- 
eting system and its reports are described later. 

Another condition concerned classroom space; the grantee was to develop and 
implement a plan "to guarantee significant excess pupil capacity among the schools 
participating in the voucher experiment, if the grantee continues in the voucher 
experiment in academic year 1973-1974.'* This condition was not satisfied and a 
supplemental grant provided a one-time allowance for trailer classroom space, (In 
the March 1973 grant, a Student Space Voucher Fund was set up. This provides a 
''space voucher** of $4 per participating student. These vouchers are pooled and 
redistributed as additional space needs for program expansion arise.) 
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Restrictions were placed on the spending of funds in several budget categories; 
the Inservive Training C(as7 could not be increased by any internal budget adjust- 
ment and the Compensatory Voucher ioidX could not be adjusted. The funds for the 
incremental bussing and planning grants to new schools could be utilized with prior 
written permission, based on written documentation of need and justification for the 
expense for the district. 

The same restriction was also placed on the funds in Student and Financial 
Accounting. This applied to the allocation of $15,000 per school in the initial grant 
and to increases in this fund as a result of supplemental grants. The district was also 
to provide a detailed description of how the $15,000 allocated to each voucher school 
(category 2.6E) to provide additional administrative assistance was being used. 

As a condition of the initial grant, the district was also to develop alternative 
plans for the second and third year of the demonstration with one of the options 
based on the passage of enabling legislation to allow expansion to a full voucher 
model involving additional public, private and/or parochial schools, and another 
option based on a phaseout of the existing internal voucher model involving the six 
voucher schools. (The second and third year budget data apply to an expansion of 
the demonstration to in Jude seven newly entering voucher schools in the 1973-74 
school year.) 

Supplemental Grants 

As shown in Table III-6, the major portion of the supplemental grants covered 
the increased cost of activities related to the administration of the voucher demon- 
stration and to inservice training. Several new items were included in the first 
supplement, but for the most part the increases occurred in thos^ categories which 
were originally understated. An attendance clerk was added to the central staflT 
because of the increased pupil accounting workload that resulted from the introduc- 
tion of a second parallel pupil accounting system. Some additional funds were added 
to continue and upgrade the existing system during the transition. Funds were 
provided for the certified audit required by the special conditions of the voucher 
demonstration grant. Salaries for substitute teachers had been omitted in the earlier 
budget. These monies were necessary to provide teacher release time for the HRC 
inservice work previously budgeted. Additional funds were provided for a part-time 
research assistant in the district's internal evaluation of the voucher demonstration. 

In the area of increased cost of activities, the compensatory voucher fund was 
supplemented by $67,000 to reflect the more accurate estimate of the number of 
students eligible for the compensatory voucher. As discussed, a one-time cost of 
trailer classroom space was included in the first supplement The contingency fund 
for bussing was increased by $1,000 in the first supplement. The other major item 
in the first supplement was an adjustment of $30,000 in indirect cost (overhead) for 
an underestimate of excess administrative load due to program implementation. 

The initial grant provided $15,000 for each voucher school for school-site ad- 
ministrative assistance; this was increased by $3,000 per school in the first supple- 
ment and by a total of $20,500 in the second supplement. As a result of these 
increases, the budgets for the second and third year of the demonstration are based 
or decreasing allowance for school-site administrative assistance. The allowance 
is .>^5,000 for the first year a school participates as a voucher school and then $20,000 
for the second year and $10,000 for the third year. If the principal wishes to maintain 
the initial higher level of effort, compensatory voucher funds and other voucher 
funds allocated to the school have to be used to augment centralized funding. 

In the first year of the demonstration each voucher school received a flat grant 
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Table HI-6 

liRtAKUOWN OK SUPPI.EMKNTAL. FKIJKR/M. CKANTS 
FOK 1972-73 SCHOOL YEAR 

(In $ thousand) 





1 ni L Id L 


Second 


Third 






rirant , 


r.rant , 


Grant , 




Area of Svipport 


hill 


12/72 


3/73 


Total 


Central admin is tra t ion 


238,7 


57,5 


137, 2 


433,4 


HRC training 


50,0 


— 


19,0 


69,0 


Ind irec t school-site suppor t 










Administration 


90,0 


18,0 


20, 5 


128,5 


Inservlce for teachers 


180, U 




100,0 


280,0 


Substitute teacher 




15,3 


43,5 


58,8 


Bussing 


10,0 


1,0 




11,0 


Subtotal 


280.0 


^4.3 


164.0 


478,3 


Dlrec t school-s it e support 










Compensatory vouchers 


4A2, 1 


67,0 




509,1 


Classroom space 




15,0 




15,0 


Subtotal 


442,1 


82.0 




524, 1 


Teacher salary guarantee 


36.0 






36,0 


Central staff salary guarantee 


30,0 






30,0 


Grants to nRw schools — community 


15,0 






15,0 


Total., all areas 


1,091,8 


173,8 


320, 2 


1,585,8 



SOURCE: Various GEO budget documontat Inn as of Marcli 1973, 



of $30,000 for teacher inservice training. Some of these funds were used by the initial 
voucher schools at the end of the 1971-72 school year as they were preparing for the 
following year In the same manner the supplement of $100,000 covers some of the 
activities of the expansion schools before July 1973. Thus, the amount of $280,000 
for inservice training in the 1972-73 budget covered the $30,000 allowance for each 
of the schools and $100,000 for the expansion voucher schools inservice training 
before the opening of the 1973-74 school year* 

In the second supplemental grant increases in severaTline items reflect the 
additional effort needed to develop the necessary management systems and to man- 
age the demonstration. Specifically, the cost of a systems analyst, an administrative 
assistant, an attendance secretary, programmers and computer support was covered 
for the part of the school year that remained. Approximately $60,000 of the $320,000 
supplemental grant was for this purpose. 

The initial grant had underestimated the general administrative impact of the 
voucher demonstration, but not by much— less than an additional 10 percent of the 
initial grant was required. The underestimate of the effort and time required to 
develop and install the management support systems was to be expected and it is 
doubtful that more money could have effectively reduced the development time. 



DISTRIBUTION OF VOUCHER DEMONSTRATION FUNDS 



The distribution of funds between the demonstration's administrative activities 
and the voucher schools provides the necessary fiscal context for discussing the 
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budgeting changes brought about by the voucher concept and the school-level impact 
of the voucher funds. Delineating the fiscal context also sheds some light on the 
interesting question of start-up versus continuation costs for the demonstration as 
a whole. The distribution, shown in Fig. III-l, includes the estimated grant for 
1973-74 because of the change in the distribution of funds between direct and in- 
direct school-site support as the size of the demonstration increased. 

The administration of the voucher demonstration includes the activities of the 
Sequoia Institute for voucher project management, the added workload of the cen- 
tral district staff, the additional staff of the central district office, additional oper- 
ating expenses (printing, travel, supplies) at the district level, and the cost of HRC 
training activities. The Professional Development Center of the second year is con- 
sidered as an administrative activity. The salary guarantee funds for teachers and 
central services staff and the planning grants for both Alum Rock schools and 
community-initiated schools are grouped together in the administration subcatego- 
ry of contingency funding. 

Direct school-site support covers the compensatory voucher funds for each mini- 
school, temporary classroom space ^trailers), and the student space voucher fund. 
Indirect school-site support includes the allocation of funds for administrative assist- 
ance to the schools, the allowance for incremental bussing, and the cost of inservice 
training and substitute teacher salaries. 

For the first year, the demonstration included six voucher schools and almost 
4000 students; the direct and indirect support of these six schools accounted for 
about 63 percent of the total funds, split about evenly between direct and indirect 



1972- 1973 



Total 
Budget 
SI, 585, 756 



■36.8%- 



Administration 
of 

Voucher 
Denx)nstration 



■63.2%- 



Voucher 
Schools 



31.7%- 
- 5.1% 

33.0% - 



Administration 



Contingency Funding 



Direct School Support 



30.2% Indirect School Support 



1973-1974 



Total 
Budget 
$2,924,868 



•34.2%' 



Administration 
of 

Voucher 
Demonstration 



29.4% ' 



Administration 



4.8%^^— Contingency Funding 



.65.8%- 



Vou cher 
Schools 



45.1% Direct School Support 



20.7% Indirect School Support 



Fig. III-l — Flow of funds to administration of the voucher 
demonstration and to the school sites 
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support. When the demonstration expanded to thirteen schools and approximately 
9000 students, the distribution of funds to direct and indirect school support in- 
creased only slightly. But the direct support of the schools, mostly the compensatory 
voucher, increased to 45 percent of the total grant and was more than twice the 
funds allocated to indirect school support. 

The funds for the second year of the demonstration were almost twice those of 
the first year — a not unexpected occurrence, since the demonstration had almost 
doubled in size (Table III-7). The interesting question, however, concerns the absence 
of an expected decrease after the start-up activities were over and the demonstration 
had entered a more operational phase. 

The answer appears to be that the voucher demonstration in the 1973-74 school 
year is still in the start-up phase; the seven new voucher schools, according to the 
funding support allocation rules, are in the same phase as the six original voucher 
schools were in the first year of the demonstration. They have, for example, the 
higher allowances for school-site administrative assistance and for inservice train- 
ing. 

Delineation of start-up versus continuation cost of the demonstration will re- 
main mostly an academic or accounting exercise until the number of voucher 



Table II1-7 
COMPARISONS OF FIRST AND SECOND YEAR GRANTS 



Budget Category 


First 
Year, 
1972-73 


% of 
Funds 


Second 
Year, 
1973-7A 


% of 
Funds 


Adminis trat ion 

District — additional activities 
District — added staff 
Added operating expenses 
Sequoia Institute 
HRC training 

Professional Development Center 


56.963 
196,438 

62,803 
117, lAO 

69,000 






10A,871 
299,057 
1A3,750 
178,112 
100,000 
35,000 






Total, administration 


502, 3AA 


31 


7 


860,790 


29,4 


Contingency funding 

Central staff salary guarantee 
Teacher salary guarantee 
Grants to new schools, community 
Grants to new schools, public 


30,000 
36,000 
15,000 






50,000 
lA.OOO 
15,000 
6^000 






To tal , cont ingency funding 


81,000 


5, 


1 


139,000 


4 




Direct school-site support 
Compensatory vouchers 
Classroom and space vouchers 
Total, direct support 


509,100 
15,000 
52A,100 


33. 


0 


1,282,A50 
36,000 
1,318,A50 


45 


.1 


Indirect school-site support 
Administrative assistance 
Substitute teacher salaries 
Inservice for teachers 
Incremental bussing 

Total, indirect support 


128,500 
58,812 

280,000 
11,000 

A78,312 


30. 


2 


375,000 
37,628 

170,000 
2A,000 

606,628 


20 


.7 


Total grant 


1,585,756 


100. 


0 


2,92A,868 


100 


.0 



SOURCE: The federal grant and related correspondence as of March 
1973, 
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schools stops increasin^^ There should be, however, some impact on costs from 
learning curves as the experience of the initial schools is transferred to the new 
schools, as the voucher-enabling management systems are developed, and as the 
expertise of the central district staffin dealing with the requirements of the voucher 
concept increases. 

The administrative assistance allowance to the schools based on their experi- 
ence in the voucher demonstration provides tangible evidence of a planned decrease 
in operational cost as the demonstration moves out of the start-up phase. The same 
type of decreasing allowance applies to inservice training; the flat allowance of 
$30,000 per school in the first year of the demonstration has now changed to a per 
student allowance, with a school receiving $50 per student for the first year of 
participation and $10 per student for the second year (no federal funds are to be used 
for inservice training after the second year). The district's effort to develop and 
install the school-level income/outgo budgeting system and the more timely and 
accurate attendance system, and to maintain dual attendance systems during the 
transition, also entails a one-time cost — estimated to be less than $75,000. This does 
not include the cost of entering the 5000 students of the second year or the school-site 
incremental administrative costs. 

The voucher component of parental participation is definitely an incremental 
cost to district operations and, at about 30 percent of administrative costs, is a 
relatively high-cost activity that will continue. Collecting and distributing adequate 
information about the choices open to parents make up the major portion of the cost 
of parental participation. But, as with the other activities, its cost should decrease 
somewhat with experience and after the procedures and techniques are developed. 
Afler the start-up period, voucher demonstration participants should have a better 
idea of the type of information needed by parents, how to collect it, when parents 
need it, and how best to get the information to them. 

Additional insights can be gained by comparing the 1973-74 grant with the 
projected grant for 1974-75 even though supplemental gra^^.ts, resulting from 
negotiations, may significantly affect the cost of specific activities. (See Table III-8.) 

The grant for 1974-75 covers a voucher demonstration of the same size as in 
1973-74; that is, ten elementary schools and three middle schools with 45 mini- 
schools and approximately 9000 students. The direct school-site support remains the 
same. The decrease of about 50 percent in the indirect school support occurs primari- 
ly because the schools are in their second and third year of participation and have 
a lesser need for inservice training and school-site administrative support. The small 
decrease of 16 percent in central administration is attributable to a reduction in the 
central district staff" time devoted to the voucher demonstration and in staff" time for 
management systems development. 
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Table Ul-8 
(;6MPAKrS0N OF SECOND A^^^ THIRD yI'::ar crants 



Budget Category 


Second 
Year, 
1973-7A 


Third 
Year, 
1974-75 


Administration 

Distric t--add itional ac t ivit ies 
District — added staff 
Added operating expenses 
Sequoia Institute 
HRC training 

Prof ess iona 1 Deve lopment Center 
Tota 1 , admin is t ra t ion 


lOA ,871 
299,057 
1A3,750 
178,112 
100,000 
35,000 
860, 790 


85 ,851 
298,805 
133,500 
188,548 

20,000 
726,704 


Con t ingency funding 

Central staff salary guarantee 
Teacher salary guarantee 
Grants to new schools, community 
Crants to new schools, public 


50,000 
1A,000 
15,000 
60,000 


30,000 


Total, contingency funding 


139,000 


30,000 


Direct school-si te support 
Compensatory vouchers 
Classroom and space voucliers 
Total, direct support 


1,282,A50 
36,000 
1,318,450 


1,282,450 
36,000 
1,318,450 


Indirect scliool-s i te support 
Administrative assistance 
Subs t i tu te teacher salar ies 
Tnservice for teachers 
Incremental bussing 

Total, indirect svipport 


375,000 
37,628 

170,000 
24.000 

606,628 


200,000 
20,532 
50,000 
2 5^00 

205,532 


Total grant 


2,924,868 


2,370,686 



SOURCE: Federal grant and related correspondence as 
of March 19 73. 



Appendix IV 

THE VOUCHER FINANCIAL INFORMATION SYSTEM 
by William S. Furry and Sue A. Haggart 



All federal voucher demonstration grants to Alum Rock have included the 
"special condition" requiring the district to develop and implement an "income/ 
outgo budget*' for the voucher mini-schools. The system was de- signed during the 
first year of the project and was in operation at the beginning of the second year. 
The income/ outgo report, to be distributed monthly to the mini-schools, displays the 
income changes, and corresponding expenditure budget changes, resulting from 
student transfers during the report period. 

In the income/outgo accounting system, each mini-school is treated as an in- 
dependent accounting entity; that is, as a budget, or cost, center with a self-balanc- 
ing budget. The report shows a mini-school's sources and amounts of income and its 
expenditures, encumbrances, and unencumbered balance. The computer program 
that produces the report automatically adjusts the mini-school's reserve account at 
the end of the period to reflect changes in projected annual basic and compensatory 
voucher income. 



WHY INCOME/OUTGO ACCOUNTING SYSTEM IS NEEDED 

Financial accounting systems are constructed to provide information for deci- 
sionmakers. In Alum Rock we find four groups of decisionmakers: the school board, 
district administrators, school staffs, and community members. The financial infor- 
mation which these groups require in a voucher education system can best be 
understood in terms of three salient characteristics of that system: decentralization 
of the design of instructional programs and the selection of resource inputs, parent 
choice, and an enrollment economy. 

Teachers are deeply involved in designing educational programs and selecting 
resource inputs. They need to know how much discretionary income is available for 
their mini-school; how much of it already has been spent; and how much is left to 
be spent. 

Parents in making their choice of instructional programs for their children need 
to know the priorities given by the mini-schools to various purposes, activities, and 
resources. In addition, parents should be able to verify that mini-school income is 
in fact being spent for the purposes advertised in promotional literature. And, 
further, parents should be aware of rising or falling enrollments as a clue to the 
satisfaction or dissatisfaction of other parents with mini-school performance. 

Finally, the enrollment economy of the voucher demonstration requires that the 
accounting system have the capacity to adjust automatically the projected annual 
income of the mini-school. Since, during the year, only estimates of the actual 
average daily enrollment for the full year are available, and since students are free 
at any time to transfer from one mini-school to another, school staffs need periodic 
reports on the income trend in order to make appropriate changes in expenditure 
budgets. 

The income /outgo system and its monthly report, described below, meet these 
O 20 

ERIC 



21 



needs of parents and school staffs. In addition, the system will furnish the financial 
information normally required by the school board, district administration, voters, 
and outside organizations and individ- uals. These needs relate to the financial 
condition and expenditure accountability of the district. Mini-school budgets can be 
aggregated to the building level and combined with the budgets of nonvoucher 
schools to provide a picture of total district activity, of the degree to which board 
policies are being carried out, and of the general financial position of the district. 



THE MONTHLY REPORT OF THE INCOME/OUTGO 
BUDGETING SYSTEM 

The periodic report for each mini-school contains three sections: an income 
report, an expenditure report, and a cost analysis section. 

The income report shows, for each income source (such as Basic Voucher, Com- 
pensatory Voucher, Model Cities, Administrative Allowance, Psychological Ser- 
vices, and so forth), the budget at the beginning of the school year, the budget at the 
beginning of the current report period, the amount of change during the period, and 
the projected annual income at the end of the period. In addition, it displays the 
amount of subsidies or contributions incurred by the mini-school. 

The expenditure report shows for each program (such as Basic Instruction, 
Music, Compensatory, Physical Education, Model Cities, and others) an account 
within the program (teacher salaries, equipment, basic supplies, and so forth), the 
budget at the beginning of the school year, the budget at the end of the current 
report period, adjustments made during the period, this-period and year-to-day 
expenditures and encumbrances, and the unencumbered balance at the end of the 
period. The report displays totals for individual programs, all programs, and the 
reserve accounts which absorb minor fluctuations in income. 

The cost analysis report is a supplementary document containing comparative 
financial data for the voucher district. Comparisions can be made of expenditure 
budgets for: 

1. Accounts with programs, 

2. Total program, 

3. Total of all programs, 

4. Reserve accounts, and 

5. Total of all programs and reserve accounts. 

Two forms of comparison are available at each of these levels of aggregation. First, 
the expenditure budgets at the end of the current period can be compared with: 

1. The budget at the beginning of the school year (the board-approved budget), 

2. A hypothetical budget based on average salaries (the expenditure report uses 
actual salaries), and 

3. A hypothetical budget based on districtwide per pupil expenditures. 

The second form of comparison available from the cost analysis report is between 
actual budgeted expenditure in a mini-school and the average actual budgeted 
expenditure of all mini-schools. Thus, for each account, program, and all programs, 
the observer can compare the level of expenditure per pupi7 budgeted in one mini- 
school with the average for all mini-schools. Finally, by looking at the reports of 
individual mini-schools, any desired comparison between mini-schools in budgeted 
expenditure per pupil can be made. 
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PRODUCTION OF THE INCOME/OUTGO REPORT 

The income/outgo report is prepared monthly, with the reporting period corre- 
sponding to the Santa Clara County fiscal calendar periods. Three financial inputs 
are required by the computer program which produces the mini-school income/ 
outgo report. First, expenditure data (encumbrances and expenditures) are obtained 
from the district's computerized expenditure ledger system. Second, the status of 
basic and compensatory voucher income of the mini-schools at the end of the report 
period is picked up from the voucher dollar allocation report (VDAR), The VDAR 
uses data generated by the Alum Rock Attendance System (ARAS), And third, 
amounts of changes in mini-school income from nonvoucherized sources are pro- 
vided by the voucher accountant. The income/outgo program generates its reports 
from these three data sets. 



IMPACT OF MINI-SCHOOL ACCOUNTING 

Alum Rock has progressed from districtwide accounting for instructional re- 
sources to the school level and now to the mini-school level. Those familiar with 
school district budget will recognize the increased information available at the 
school level, as shown in Table IV-1, The district, moreover, is as close as any in the 
nation to accounting for resources by classroom via the mini-school budgets. 

Financial voucherization of the district has evolved slowly. During the first year 
a good deal of effort went into developing resource allocation rules and values. These 
rules, summarized in Table IV-2, are the basis fur determining the school and 
mini-school initial income and income changes during the year as a result of student 
transfers. 

These changes have not been easy. The number of budget centers has leaped 
from one to more than ninety, Districtwide number of purchase orders to be proc- 
essed has increased in three years from 3000 to more than 10,000, The rules govern- 
ing resource allocation have become so complex that it has been impossible to 
develop standard forms for school staffs to compute their own budgets. The district 
has had to employ an experienced systems analyst to design and install new student 
and financial accounting systems. And a new position in the hierarchy, that of 
Controller, has been created to head the accounting department and revamp the 
purchasing and payroll systems. 

The effect on the district of the availability of financial information at the 
mini-school level will be studied in detail by Rand. At this time, we will suggest two 
possible consequences of significance. 

First, the existence of financial data for mini-schools may make an issue of the 
role of the district's Board of Trustees in school and mini-school expenditure deci- 
sions. Board members, at the final budget session in August 1973, in fact, asked 
whether, by approving the state-required budget document (which is a function and 
object classification for the district as a whole), they were also giving their approval 
to every mini-school budget. 

Second, the financial data at the mini-school level may highlight the relation- 
ship between money and the nature of the instructional program that can be pro- 
vider]. The educational value of different kinds of personnel, materials and equip- 
ment, and support services may be increasingly questioned. The fact that something 
must be given up in order to purchase something else may be more clearly realized 
at the mini-school level and teachers may seriously consider alternative ways to 
achieving goals. 
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Table TV-2 

VOUCHER DKMONSTRATION RESOURCE ALLOCATION VALUES AND BASES, 
J972-73 AND 1973-74 



Income Source 


1972-73 


1973-74 


1 . 


Basic Voucher 


K-6: $787.86/pupil^ 
7-8: $104l.38/pupil 


K-6: $842.63/pupil 
7-8: $1047/38/pupil 




Less Central 
Services Payback 


$259 53/DUDil 


$289. 63/pupil 


2. 


Compensatory Voucher 


K-6: $237.77/eligible 
7-8» $301 55/pli2ible 


K-8: $275/eligible 
pup-l.lb 


3. 


Administrative 
Allowance 


$18,000 flat grant for 
each school 


First year, $25,000 
Second year, $20,000 


4. 


Inservice Allowance 


$30,000 


First year, $50/pupil 
Second year, $10/pupil 


5. 


Psychological Services 


$6.73/pupil 


$7.75/pupil 


6. 


Curriculum Coordinators 


$7.46/pupil 


$10.73/pupil 


7. 


Educational Media 


$4.76/pupil 


$5.68/pupil 


8. 


Curriculum Voucher 


$30/Program Teacher 


$30/Program Teacher 


9. 


Carryover Funds 


None (except inservice 
al lowance— see No. 4) 


Funds not spent 1972-73 
may be carried over. 


10. 


Categorical Income 

^T^t-1^a T Mn/-? q1 

^iicie 1, noQei 
Cities, SB 90, 
Title VII, etc.) 


Flat grants to schools allocated on basis of 
economic disadvantage, educational need, and for 
continuation of programs already in existence 
in schools. 


11. 


State Excess Cost for 
Education of Special 
Students 


Special education 
programs budgeted 
centrally . 


Allocated to special 
education mini-schools on 
a per-class basis. Amount 
varies by program. 


12. 


Special Education 
Subsidy 


Special education 
programs budgeted 
central ly . 


Allocated to special 
education mini-schools to 
pay for district mandated 
addi t ional personnel . 


13. 


Basic Program Subsidy 
or Contribution 


A reduction of or addition to basic voucher income. 
Equals the difference between actual salaries of 
basic program personnel and the average salary cost 
of these personnel. 



^Pupil in average daily enrollment. 



Mayfair School receives $50/eligible pupil and a flat Title I grant. Stu- 
dents in Conniff*s Title VII program are not eligible for compensatory vouchers. 



Appendix V-A 
CLASSROOM OBSERVATION CHECKLIST 



1 . Academic \ 

2. Artistic - 

3. Recreational/Socia 

4. Procedure 



ACTIVITY SEQUENCE 



Time 


Type 















































































































































25 



26 

CLASSROOM RESOURCES 



In-Use Resources 





individual 


1 

Group 


Self- j 
Selected 'Directed 




Est Imated 
Quantity 


Stored 


AiiJio/Vi sual 
















Kucords; phonographs 














1 


Recorders 














1 


Musical inst ruijiciits 














1 


SI i Jo projector 
















Movie projector 
















Cards 
















riannclboards 
















Maps, cliarts 
















P ictures 
















Spec i mens 
















Plants 
















Pets 
















Science materials 
















Toys, games 
















t)vcrhead projector 
















Art sup]) lies 
















Manipulative devices 
















i ilii:strii)s 
















Kii.ds of Books 
















Reference 
















I ext l)ooks 
















V'OTkbooks 
















Prottrammed Texts 
















"iTee reading" 
















Supplementary text 
















"Keaders" 
















l-ncycloi)edi as 
































Commercial Materials 
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INSTRUCTIONAL STRATEGY 

Language Social Recrea- 
, Reading Arts Math Studies Science tional Arts Mus 1 c 

St af t i lu^ ^ 

S 1 1 ^:lc icac}\er 

. tca ci i in^ i i 

\ 1 do s 

't\<. r tutor 1 

- I 

{'iT( Uj^ s 

1 [ ^ I\U 1])S I 

i''^ - 'iy'^y ^jil^ : 

i^;_^idual 

^j-'-i^'Ji.''^ Acti vities 

Vyj' k IV o r k ^ J ^ 

Discussions i 

r»amos 

\^ 1 ackboar d 

__J12^ju ia>;, d rawing 

1 ^0 c 1 1 ing 

S c«2^i,_ in to nnat ion 

Mov,' about ruoia 

iiidt /-w-n dent work 

i'ru .>'cts 

__S 1 n fi J Jllii^t. A^-'^t rtimcnt 

j'-^Viti aloud ) ^ 

Takes quizzes 

i c tu . /\y 1 1 V i 1 1 c s 

i,c c^ u res I 

y^ik t s e xp lana t i on 

I ntroduces Subject , ^ . 

Rca< to class 

_J)riJ 

R'._vjcu lu. i ter ial 

Ques I i ons : fa ctual 

" ope.. - • ended ^ 

_ ^ iLr!ili£l'-^ - 

(iivo,^ demonstration . , 

i\i isv^ '.- rs question s . 

Inlr o duces, leads game , 

S hows video tape film ^ 

Plays record | I ^ 



Appendix V-B 



INFORMATION PACKAGE FOR MEETINGS WITH 
MINI-SCHOOL FACULTIES IN THE SPRING 



School: Program: ^ 

Grade Levels In Classroom; . 

FORM 1— DOCUMENTATION OF MAJOR PROGRAM CHANGES 

Please llst> In chronological order > major changes In your program 
since September oi 1972. For each change^ give the nature of the change > 
the approximate date, and the reason for the change. One or two sen- 
tences should be enough; you will have an opportunity to discuss the 
changes In more detail during our meeting. 
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School: Program: 

Grade Levels In Classroom: 

FORM 2— ESTIMATED ALLOCATION OF CLASSROOM TIME 

For each subject, please give an estimated time per week. If you don't teach 
the subject, fill-in a zero. If you need more subject spaces," divide a space 
Into two with a slash. For example, if there is a separate health studies 
period — show Science/Health and 90/AO for the time. 

Please remember that this is student Instructional and activity time. If you 
have students on a staggered schedule, give the estimated time for only one 
group . 



^^\^Graties 




Estimated >liuui;es per V,*eek by Grade 


Su'ojcct Arcas^.^ 


K 


1 


2 


3 


4 


5 


6 


7 


8 


Lfla,^.ua)^ot Arts 




















Rei'di 11 g 




















Af 3 thmo tic 




















loci.^l Studios 




















Sc J cucc 




















Act 




















Mus Ic 




















i*i:o;^iaT> Activity ^^'^ 
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(a) Includes: Oral/written expression, spelling, handwriting, foreign languages. 

(b) Briefly describe on the back of this page the nature of the activities. 

(c) Includes: Opening exercises, lunch, pupil-^teacher planning. 
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School: Program: 

Grade Levels in Classroom: 

FORM 3— MAJOR INSTRUCTIONAL MATERIALS USED 

Please list the major instructional materials used. (As a criterion, 
consider importance rather than dollars spent.) Give publisher and/or 
author, if applicable — e.g., BRL-SILAS or Mayne/Ferguson Math Program 
and indicate grade levels where materials are used. 



READING MATHEMATICS 
MATERIALS GRADES MATERIALS GRADES 



OTHER SUBJECTS— SPECIFY SUBJECTS 
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School: - Program: 

FORM ^4— 'FIELD TRIPS AND SPECIAL PROJECTS 

Please list the field trips you have taken this year, including those 
planned for the rest of the year. Show grade levels involved and give 
nn estimate of time — 2 hours, half a day, all day. 

FIELD TRIP DURATION GRADES 



For special projects or activities, briefly describe the project, the grades 
involved, the. number of students participating (the whole class or different 
projects for different groups of students). 



Appendix V-C 

EDUCATIONAL COSTS BY SCHOOL, MINI-SCHOOL, AND 

CLASSROOM 

by Sue A. Haggart 

SCHOOL AND MINI-SCHOOL BUDGET LEVELS 

The voucher demonstration started before the budget machinery to support it 
was ready. The year before the demonstration, however, was a year of phasing into 
a partially decentralized school system and the voucher demonstrcition funding 
carried with it a special condition that a school level budgeting system be developed. 
During the 1972-73 year, therefore, the information at the school level v/as caught 
in the middle between the conventional line-item district budget format and the 
emerging school level budget design. Budget information at the mini-school level is 
even more elusive, but is worth retrieving in order to establish a mini-school budge- 
tary baseline for the 1972-73 voucher demonstration (see Table V-C.l and V-C.2). 

The school level budget includes funds for the basic instructional program (I'ro- 
gram 111 for elementary school students and Program 141 for middle school stu- 
dents), a subject allocation for middle school students, categorical aid programs, 
centralized services allowance, and for the voucher schools. Program 120 — Internal 
Voucher School Program. The Internal Voucher School Program money is the 
budgeted amount for the compensatory voucher and the voucher-provided adminis- 
trative allowances to the schools. 

The Program 111 funds for voucher elementary schools in 1972-73 show a medi- 
an of $472 per student — considerably lower than the $535 median of the non voucher 
schools— and a range of $458 to $637 per student. Pala has a per student amount 
of $683 fur Program 141— slightly higher than the median of $669 for the nonvouch- 
er middle schools. The differences are not related to either school size (a larger 
number of students yields a lower per student figure) or to the salary differences of 
the school's teaching staff. Rather, the differences reflect the district policy of equal- 
izing total funds for each school. In 1973-74, Mayfair and Coniff eligible students will 
receive a "discounted" voucher and more SB 90 state funds will be allocated to 
schools not receiving funds from other services. 

Because of differing proportions of students eligible for the compensatory vouch- 
er, the dollars per student for the twenty-two mini-schools ranged from $586 (Meyer 
Basic Skills) to $969 ^Miller Individualized) (see Fig. V-C.l j. The data illustrating the 
variation in dollars per student for the schools and mini-schools are subject to a very 
real limitation. Given the state of the art in conventional budgeting practices, 
determining a dollar per student factor by examination of the records at any time 
after the figures were initially prepared can be in error. The error arises because 
the exact enrollment figures on which the estimates are based are not carried along 
with the budget estimates in later publication of the data. Ordinarily, this presents 
no problem because of the one-time nature of the initial calculations and because 
a "backward look** at the data is rarely taken. Here, the dollars per student for 
Miller could be overestimated because of the mismatch of the student enrollment 
and the budget figure. The point to be made is that there is a difference in the dollars 
per student for the mini-schools and the absolute difference is not of great impor- 
tance. 
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Table V-C.l 

DISTRIBUTION OF FUNDS TO ALUM ROCK ELEMENTARY SCHOOJ.S, 1972-73 
(In $ per regular student) 



School 


To ta 1 
$ per 
Student 


El enientary 
Program 111 


Title I 


CoiTipcn sa to ry 
Voucher^ 


Other^ 


Title I schools^ 












San Antonio 


915 


623 


292 




— 


Arbuckle 


816 


538 


2A6 




32 


Hubbard 


782 


53A 


2A8 






Mayfalr 


72A 


A81 


2A3 






Slonaker 


722 


AA9 


2A0 




33 


Med Ian 












Voucher schools 












Cassell 


7 OA 


523 




168 


13 


Goss 


682 


A58 




179 


A3 


McCollam 


669 


472 




195 


2 


Meyer 


662 


A55 




159 


A8 


Miller 


898 


637 




236 


25 


Median 




A72 








Nn n Vn 1 1 r h p T* 












non-Title I schools 












Painter 


65A 


6A9 






5 


Dorsa 


J 


538 






25 


Cureton 


':>':)l 


5A9 






3 


Ryan 


552 


552 








Linda Vista 


55A 


533 






20 


Rogers 


503 


500 






3 


Shields 


A63 


A61 






2 


Lyndale 


A52 


A49 






3 


Median 




535 









The elementary level (K-6) compensatory voucher was $237.77 for stu- 
dents eligible tor the free lunch program* The cost per student shown 
here Is based on total enrollment of regular students. For Miller School, 
almost all the students apparently are eligible for the free lunch program. 



Model Cities and Community School programs. 
^Conlff, a Title VII school, Is not Included. 
Mt. Hamilton, with less than ten students. Is not Included. 



Table V-C.2 

DISTRIBUTION OF FUNDS TO ALUM ROCK MIDDLE SCHOOLS, 1972-73 
(In $ per regular student) 





Total 


lAl-152 Middle 




Middle 






$ per 


School Plus 


Compensatory 


School 




School 


Student 


Extracurricular 


Voucher^ 


Library 


Other^ 


Nonvoucher 












Mathson 


893 


821^ 




18 


5A 


Sheppard 


716 


688 




15 


13 


George 


670 


649 




21 




Fisher 


665 


630 




15 


20 


Median 




669 








Voucher — Pala 


859 


683 


153 


23 





The middle school level cumpensatc»ry voucher was $301.55 for stu- 
dents eligible for the free lunch program. The cost per student is 
based on total enrollment of regular students. At Pala, 387 students 
out of 601 apparently were eligible. 



Model Cities and Community Schools programs. 

^Mathson has a "special allowance" of approximately $90,000 or $138 
per student. 
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Mini-school Code 



F 



• 

N 
E 
B 
G 



• 

L 
A 
H 
C 
K 



• 

Q 
J 
u 

D 



• 

W 
P 
T 



_1_ 



J- 



V 
R 



625 650 20 700 20 750 20 800 20 
Dollars per regular student 



850 20 



850 



A 


Cultural Arts 


B 


Daily Living 


C 


Traditional Plus 


D 


Kindergarten 


E 


Activiiy-Centered 


F 


7th Grade School 


G 


Develop. Reading 


H 


TroditioTKil 


J 


Enrichment 


K 


Continuous Progress 


L 


Individ. Learning 


M 


Basic Skills 


N 


Sullivan - BRL 


P 


Fine Arts — Meyer 


Q 


School 2000 


R 


Individ- Study 


S 


Multi -cultural 


T 


Academic Skills 


U 


Three R's Plus 


V 


Creative Arts 


w 


Fine Arts — Palo 


X 


Math /Science 



Fig. V-C.l — Distribution of mini-schools, by dollars 
per regular student 



An examination of the mini-school budgets with the voucher host-school also 
reveals some interesting variations (see Fig. V-C.2 and Table V-C.3). Here the varia- 
tions can be explained, for the most part, by variations in the enrollment of both the 
host school and the mini-school and in the per pupil difference for building personnel 
for the mini-school. 

The unembellished, cold totals for the basic voucher and the compensatory 
voucher shown in the mini-schools' budgets do not tell the story of the voucher 
demonstration's impact at the school level. The voucher allocation rules may give 
the impression that everything is set in concrete. This was not the case even in the 
first year. Voucher schools and mini-schools did have discretionary money; they did 
differ in what they spent the money for. The voucher schools had the conventional 
$8.50 per student allocation for instructional supplies. This, in effect, is the discre- 
tionary money for the school as a whole; and, along with the basic voucher funds, 
the instructional supplies allocation is under the control of the principal. The man- 
ner in which the voucher school's administrative allowance was to be spent was also 
under the control of the principals. The compensatory voucher funds and some of 
the inservice training and planning funds were under the control of the mini-school 
faculties. 

It should be noted that the "compensatory" funds were not earmarked for 
compensatory activities, such as remedial reading, or for specific students. One 
mini-school faculty elected to spend some of its discretionary money to hire a part- 
time teacher for its program. Another voucher school decided to keep two "excess" 
teachers by paying their salaries from their discretionary pot of money. In these 
instances, the schools and mini-schools were decreasing the student-teacher ratio as 
an option they elected to take. A mini-school is not allowed to increase the student- 
teacher ratio, however, as a way of gaining discretionary money. 
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Table V-C.3 
SCHOOL AND MINI-SCHOOL BUDGET DATA, 1972-73 



1. Cassell 







A 


B 


C 


D 








Budget 






Tradi- 










Program 


Cultural 


Daily 


tional 


Kinder- 




Category 


Code 


Arts 


Living 


Plus 


garten 


Total 


Number of students 




159 


195 


217 


112 


683 


Allocation percentages 




23 


27 


32 


. 18 


100 


Internal Voucher School 


120 


$ 25,883 


$ 30,386 


$ 36,103 


$20,257 


$112,629 


$/student 




163 


156 


166 


181 


165 


Elementary School 


1-1 


80,676 


9^,707 


112, 2A5 


63,137 


350,767 


$/student 




507 


A86 


513 


56A 


51A 


Model Cities 


119 


2,075 


2,A35 


2,887 


1,62A 


9,021 


Total budget 




108,6 iA 


127,528 


151,235 


85,018 


A72,A15 


$/student 




683 


65A 


697 


759 


692 


2. Goss 






E 


F 


G 










Budget 
















Program 


Activity- 


7th Grade 


Developmental 






Category 


Code 


Centered 


School 


Reading 


Total 


Number of students 




322 


91 




301 




71A 


Allocation percentages 




A5 


12 




A3 




100 


Internal Voucher School 


120 


$ 56,917 


$15,096 


$ 5A,A68 


$126, A81 


$/student 




177 


166 




181 




177 


Flementary School 


111 


1A5,971 


38,926 


139, A8A 


32A,381 


$/student 




A53 


4 28 




A63 




A5A 


Model Cities 


M9 


6,139 


1,637 


5,866 




13,6A2 


Total budget 




209,027 


55,659 


199,818 


A6A,50A 


$/student 




6A9 


612 




66A 




650 



3. McCollam 







H 


J 


K 


L 






Budget 








Indivi- 






Program 


Tradi- 




Continuous 


dualized 




Category 


Code 


tional 


Enrichment 


Progress 


Learning 


Total 


Number of students 




ISA 


81 


1A7 


191 


603 


Allocation percentages 




33 


7 


27 


33 


100 


Internal Voucher School 


120 


$ 35,533 


$ 9,81A 


$ 31,100 


$ 37,322 


$113,269 


$/student 




193 


121 


212 


195 


189 


Elementary School 


111 


90,792 


19,259 


7A,286 


90,793 


275,129 


Mentally Gifted Minor 


177 




30,886 






30,886 


$/student 




A93 


619 


505 


A75 


507 


Model Cities 


119 


516 


110 


A23 


516 


1,565 


Total budget 




126,8^1 


6O;076 


105,808 


128,631 


A21,3A9 


$/student 




689 


7A2 


720 


675 


699 
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Table V-C. 3— Continued 



4 , Meyer 





Bud pet 


M 


N 


p 


n 






Program 


Basic 


Sullivan 


Fine 


School 




Category 


Code 


Skills 


BRL 


Arts 


2000 


Total 


Number of students 




366 


154 


117 


123 


760 


Allocation percentages 




43 


20 


19 


18 


100 


Internal Voucher School 


120 


$ 52,944 


$24,825 


$23,584 


$22,748 


$124,101 


$/s tudent 




143 


161 


202 


/ 185 


163 


Elementary School 


111 


152,907 


71,119 


65,564 


64,007 


355,596 


$/s tudent 




418 


462 


577 


520 


468 


Model Cities 


119 


8,621 


4,010 


3,809 


3,608 


20,048 


Total budget 




214,472 


99,954 


94,956 


90,363 


499,745 


$/ student 




586 


649 


812 


735 


656 



5. Miller 





Budget 


R 


S 


T 










Program 


Individual 


Multi- 










Category 


Code 


Study 


Cultural 


Academic 


Total 


Number of students 




141 


193 


135 




469 


Allocation 




33 


39 


28 




100 


Internal Voucher School 


120 


$ 36,537 


$ 44,882 


$ 26,633 


$108,052 


$/student 




259 


234 


197 




230 


Elementary School 


111 


96,302 


113,812 


81,711 


291,825 


$/st-udent 




683 


590 


605 




622 


Model Cities 


119 


3,838 


4,535 


3,256 


11,629 


Total budget 




136,677 


163,229 


111,600 


411,506 


$/s tudent 




969 


846 


827 




877 


6. Pala 




Budget 


U 


V 


W 




X 






Program 


Three R's 


Creative 


Fine 


Math/ 




Category 


Code 


Plus 


Arts 


Arts 


Sc lence 


Total 


Number of students 




211 


217 


111 




68 


607 


Allocation percentages 




32 


39 


18 




11 


100 


Internal Voucher School 


120 


$ 27,271 


$ 35,792 


$16,521 


$12,191 


$ 91,775 


$/student 




129 


165 


149 




179 


151 


Middle School 


141 


74,773 


91,129 


4n,060 


25,703 


233,665 


$/ student 




354 


420 


379 




378 


385 


Other Middle School 


142-151 


55,606 


67, 771 


31,279 


19,114 


173.771 


$/ student 




264 


312 


282 




281 


286 


Total budget 




157,651 


194,692 


89,860 


57,008 


499.211 


$/student 




74? 


897 


810 




838 


822 
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COMPARABLE MINI SCHOOL INSTRUCTIONAL COST PER 
CLASSROOM 

Inevitably, a cost per student for each of the twenty-two mini-schools will be 
calculated by various people for different purposes. One of the most important 
purposes, perhaps, would be to get an idea of the cost of a particular combination 
of resources that seems to work. The twenty-two mini-schools of the 1972-73 voucher 
demonstration do represent alternative ways to provide elementary education. The 
expectation of voucher proponents is that in the long-run, one of the outcomes may 
well be a more effective education. If this should prove to be true, then the next 
questions to be answered are: "Can the alternative be afforded?"; "How does the cost 
of the more effective alternative compare with other alternatives?" 

If budget data are used, the cobt will be biased by such things as the size of the 
mini-school and the actual teacher salaries. The objective here is to explore the 
development of another measure — the instructional cost per classroom. That cost, 
estimated for each mini-school, provides a more useful picture of alternative levels 
of* resource-use. 

As defined, the instructional cost per classroom includes classroom teacher, 
resource teacher, and instructional aide salaries (with fringe benefits, as applicable), 
instructional materials, equipment and supplies (consumables), and the cost of field 
trips. It does not include a prorated share of the noninstructional expenses of run- 
ning the school, such as the principal's salary, the secretary *s salary, clerical ser- 
vices, or custodial services. The exclusion of these expenses is an important feature 
of the measure; the instructional cost per classroom becomes a measure of the 
resources involved in the educational process at the classroom level. This assumes, 
of course, homogeneity of classroom resource-use within the mini-school. 

Another feature of the measure is its use of across-the-board average salary 
figures for the staff involved. This avoids differences among mini-schools that are 
attributable solely to salary differentials rather than to resource mix or levels of use. 
The resulting instructional cost per classroom provides a more equitable basis for 
the mini-school program cost comparisons that are bound to occur. 

The instructional cost per classroom can also be used as a basis for calculating 
a cost per student for the mini-school by assuming thirty students for each class- 
room. The cost per student derived in this manner is insensitive to minor variations 
in the number of students in the mini-school (its size) and serves as an index in 
comparing the different mini-schools. In this way, the circularity implicit in the 
conventional practice of estimating the cost per student can be avoided. Recall the 
manner in which the principal's salary, for example, was allocated to the mini- 
schools of the host school. In the case of teacher salaries, the total is divided by the 
number of students in estimating the cost per student. In the real world, if the 
teacher salary total is $91,997, then the cost per student for, say, 224 students is 
$411 — a figure that changes as the enrollment increases up to the point where an 
additional teacher is needed. If the enrollment drops to 203, the cost per student for 
teachers is $453 but an increase to 245 students would probably necessitate adding 
a teacher. The class size would be too high, in light of state requirements, at 35 
students per classroom, but the cost per student would be a much lower $375. 
Number-juggling aside, the point is that the use of the instructional cost per class- 
room and a constant 30 students for each classroom provides an index-like measure 
when the cost per student is used as a basis for comparing mini-schools. 

The instructional cost for each mini-school, the instructional cost per classroom, 
and the cost per student are all based on the cost of items the mini-school uses in 
providing its program. It is, in essence, the cost of the package of instructional 
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services unique to a particular mini-school. As mentioned earlier, the noninstruc- 
tiona! expenses of running the school site are not included. Neither is the cost of 
space. Excluding the cost of space deserves a separate discussion. 

The objective is to measure the resources that make one mini-school different 
from another in carrying out an instructional program. Only the public school 
system was involved in the 1972-73 school year, and the mini-schools represented 
only a reorganization of the existing resources of the district. All students were 
provided with **space" on a uniform basis. There are no significant variations in the 
manner in which the mini-schools use space. Including the cost of space in the 
estimate of* a mini-school's instructional cost per classroom does not change the 
relative ranking of a mini-school. In an accounting sense, however, the cost of space 
would be important and, as private schools entered the voucher demonstration, the 
cost of space would be a factor in assessing the resource-use characteristics of alter- 
native mini-schools. 

In developing the estimates of the instructional cost per classroom, it is further 
assumed that all classrooms are furnished in much the same way and that a similar 
package of basic instructional materials is provided for each classroom. These as- 
sumptions seem to be valid except for the McCollam Enrichment mini-school. This 
mini-school is also supported by state funds under the Mentally Gifled Minor pro- 
gram; in addition, budgetary data show a higher per student allocation for some of 
the central district services and supplies. 

The estimate provided here for the Enrichment mini-school may be low because 
actual resources are used in developing the estimate and because this mini-school 
had very few students eligible for compensatory vouchers. This mini-school faculty 
also refused to provide any information about the materials being used, about field 
trips taken, or about other details of their operation. The Enrichment mini-school 
faculty's claim to the title of "the cheapest mini-school" may be true. There is no 
reasonable basis for increasing the estimate. It could be conjectured, however, that 
the Enrichment classrooms started with more materials and equipment than did the 
other mini-school classrooms. They were identified as the classrooms for the district- 
designated mentally gifled minors in the years before the voucher demonstration. 

The details of the estimated instructional cost per classroom for each mini- 
school, shown in Table V-C.4, provide supporting information about the differences 
in resource-use among the mini-schools. 

In estimating the instructional cost per classroom, aide-use is calculated in 
3-hour equivalents. For example, a 6-hour aide is the equivalent of two 3-hour aides. 
This technique is necessary and logical in developing an index measure, or compara- 
ble cost estimate, for the mini-schools. In the financial context of the district, it is 
not an acceptable technique because the 3-hour aide does not receive fringe benefits 
but the 6-hour aide does. Assuming a fringe benefit package of twenty percent of base 
salary, two 3-hour aides can be employed for twenty percent less than one six-hour 
aide. This example serves to highlight the non-accounting/budgeting nature of the 
instructional cost per classroom estimates. The estimates are intended as compara- 
ble measures of resource-use by the various mini-schools, and by design do not reflect 
the realities of financial accountability. 

The instructional cost per classroom and cost per student for each of the mini- 
schools are shown in Table V-C.5 and Fig. V-C,3. The instructional cost per class- 
room ranged from a low of $17,312 for the McCollam Enrichment mini-school to a 
high of $22,315 for the Pala Fine Arts mini-school. When only the elementary grades 
(K-6) are considered, the Meyer Sullivan-BRL mini-school with $21,874 has the 
highest instructional cost per classroom. The instructional cost per classroom pro- 
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Table V-c:.^ 
(:»)MPAK\BI.K MINI-SCHnnl. INSTRUCTIONAL COSTS 



I. Cassell 



Expenditure Category 


Average 
Sa^ary^ 

($) 


Cultural Arts 
A (159 St) 


Daily Living 
B (195 St) 


Traditional Plus 
C (217 St:) 


Kindergarten 
D (112 St) 


No. 


($) 


No. 


($) 


No. 


($) 


No. 


($) 


Teachers 


16,200 


5 


81,000 


6 


97,200 


7 


113,400 


4 


64,800 


Aides (1 '-^ 3Hr-aidc) 


2,500 


5 


12,500 


1 


2,500 


2 


5,000 


1 


2,500 


Irvstruetional m.iterials 






4,044 




3,967 




4,53.9 




2,409 


Instructional, supplies 






4,379 




2,577 




1,625 




415 


Instructional equipment 






2,187 




2,416 




1,370 




175 


Otlu'r costs of instruction^ 






1,781 




1,837 




1,875 




941 


Mini-school instructional cost 






105,891 




110,497 




127,789 




71,240 


Instructional cost/classroom 






21,178 




18,416 




18,256 




17,810 


Cost per student'^ 






706 




614 




609 




594 



2. Goss 



Expenditure Category 


Average 
Salary^ 

($) 


Ac tivi ty-Centered 
E (322 St) 


7th Grade School 
F (91 St) 


Develop. Reading 
G (301 St) 


No. 


($) 


No, 


($) 


No. 


($) 


Teachers 


16,200 


10 


162,000 


3 


48,600 


9 


145,800 


Aides (1 = 3Hr-aide) 


2,500 


7 


17,500 


2 


5,000 


5 


12,500 


Instructional materials 






4,736 




1,147 




3,464 


Instructional supplies 






3,490 




535 




1,837 


Instructional equipment 






3,952 




561 




2,665 


Other costs of instruction^ 






6,410 




4,373 




5,590 


Mini-school instructional cost 






198,088 




60,216 




171,856 


Instructional cost/classroom 






19,809 




20,072 




19,095 


Cost per student*^ 






660 




669 




637 



3. McCollam 



Expenditure Category 


Average 
Salary^ 

($) 


Traditional 
H (184 St) 


Enrichment 
J (81 St) 


Continuous Progress 
K (147 St) 


Indiv. Learning 
L (191 St) 


No, 


($) 


No. 


($) 


No, 


($) 


r , 


($) 


Teachers 


16,200 


6 


97,200 


3 


48,600 


5,5 


89,100 


6 


9;,.?oo 


Aides (1 = 3Hr-aide) 


2,500 


2 


5,000 






3.3 


8,250 


1 


2,S00 


Instructional materials 






4,400 




2,636 




3,127 




11 ,623 


Instructional supplies 






666 




160 




941 




772 


Instructional equipment 






2,767 




399 




990 




4,462 


Other costs of instruction 






656 




141 




547 




656 


Mini-school instructional cost 






110,689 




51,936 




102,955 




117,213 


Instructional cost/classroom 






18,448 




17,312 




20,591 




19,536 


Cost per student*^ 






615 




577 




686 




651 
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Tabic V-C.A — Continued 



Expenditure Category 



Average 
Salary^'* 

($) 



4 . Meyer 

Basic Skills 
M (366 SL) 



No. 



($) 



SuUlvan-BKL 
N (154 St) 



No. 



($) 



Fine Arts 
P (117 St) 



($) 



School 2000 
Q (123 St) 



(S) 



Teachers 

Aides (1 = 3Hr-alde) 
Instructional materials 
Inst rue tional supplies 
Instruc t ional equipment 
Other costs of instruction*' 

Mini-school instructional cost 
Instructional cost/classroom 
Cost per student^ 



16.200 
2.500 



12 
5.5 



194.400 
13,750 
9,923 
4,290 
3,996 
1.645 

228,004 
19,000 
633 



5 

4.5 



81,000 
11,250 
12,834 
1,313 
2,375 
600 

109,372 
21,874 
729 



64,800 
5,000 
3,156 
1,217 
4,118 
645 

28,936 
19,734 
658 



64,800 
7.500 
2,825 
2,222 
1,066 
1,860 

81,073 
20,268 
676 







5. 


Miller 












Average 
Salary^ 


Individual. Study 
R (141 St) 


Multi-Cultural 
S (193 St)- 


Academic Skills 
T (135 St) 


Expenditure Category 


($) 


No. 


($) 


No. 


($) 


No. 


($) 


Teachers 

Aides (1 •= 3Hr-aide) 
Instructional materials 
Instructional supplies 
Instructional equipment 
Other costs of instruction^ 


16,200 
2,500 


6 
2 


97,200 
5,000 
1.419 

143 
2,096 

750 


7 

4.5 


113,400 
11,250 
3,128 
1.745 
1,568 
2,000 


5 
2 


81,000 
5,000 
1,061 
940 
368 
300 


Mini-school instructional cost 
Instructional cost/classroom 
Cost per student^ 






106,608 
17,768 
592 




133.091 
19,013 
634 




88.669 
16,200 
540 




6. 


Pala 













Average 


Three R* s Plus 


Creative Arts 


Fine Arts 


Math/Science 




Salary^ 


U (211 St) 


V (217 St) 


W (111 St) 


X 


(68 St) 


Expenditure Category 


($) 


No. 


($) 


No. 


($) 


No. 


($) 


No. 


($) 


Teachers 


16,200 


6 


97,200 


7 


113,400 


3 


48,600 


2 


32,400 


Resource teachers^ 






16,038 




18,954 




8,262 




5,346 


Aides (1 « 3Hr-aide) 


2,500 






5 


12,500 


2 


5.000 






Instructional materials 






7,525 




2,230 




1,040 




2,344 


Instructional supplies 






1,715 




3,287 




1,119 




213 


Instruc tional equipment 






722 




1,172 




2,395 




139 


Other costs of Instruction*' 






2,535 




1,170 




540 




330 


Mini-school instructional cost 






125,735 




152,713 




66,956 




40,772 


Instruc tional cost /classroom 






20,956 




21,816 




22,315 




20,386 


Cost per student^ 






699 




727 




744 




680 



NOTE: 



cost per classroom and 30 students in a classroom, is estimated to provide an equitable measure of 
a mini-school's use of resources. 



Includes fringe benefits. 
^Includes cost of field trips and contract Instructional services. 
Based on 30 students per classroom. 

d 

Pro-rated share of the Resource Center s teachers. 
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Table V-C.5 

MINI-SCHOOL INS1RUCTI0N/\L COSTS PER CIJVSSKOOM, COST PKR STUDENT, 
AND DOLLAR DIFFERENCE WI'HilN HOST-SCHOOL 



Mini-school 


Instructional 
Cost per 
Classroom 


Instructional 
Cost per 
Student^ 


Dif f erence 

Between 
Highest and 
Lowest Cost 
Mini- school 


SdSSe t 1 

Cultural Arts 
Daily Living 
Traditional Plus 
Kindergarten 


$21,178 
18,416 
18,256 
17,810 


$706 
614 
609 
594 


$ 97^ 


'/O s s 

Ac t ivi ty-Cen tered 
7th Grade School 
Developmental Reading 


$19,809 
20,072 
19,095 


$660 
669 
637 


$ 82 


M('*C('^ 1 1 mn 

i i O V' L^* L' L' \JJ ' t 

Traditional 
Enrichment 
Continuous Progress 
Individualized Learning 


$18,448 
17,312 
20,591 
19,536 


$615 
577 
686 
651 


$109 


Meyer 

Basic Skills 
Sullivan BRL 
Fine Arts 
School 2000 


$19,000 
21,874 
19, 734 
20,268 


$633 
729 
658 
676 


$ 96 


Miller 

Individual Study 
Multicultural 
Academic Skills 


$17,768 
19,013 
17,734 


$592 
634 
591 


$ 43 


Pa la 

Three R*s Plus 
Creative Arts 
Fine Arts 
Math-Science 


$20,956 
21,816 
22,315 
20,386 


$699 
727 
744 
680 


$ 64 



NOTE: These are not actual costs. The cost per student, based 
on a comparable instructional cost per classroom and 30 students in 
a classroom, is estimated to provide an equitable measure of a mini- 
school's use of resources. 



^Based on 30 students in a classroom, not on actual enrollment. 

^The Kindergarten mini-school is excluded from this range of 
d if f erences . 



vides, on an equitable and supportable basis, additional information useful in look- 
ing at the differences among mini-schools. 

The instructional cost per student, on a basis of 30 students in a classroom, 
provides insights into the differences in resource-use among the mini-schools within 
each school. The widest variation was within McCollam, where the highest mini- 
school spent $109 more than the lowest mini-school. Variations within Cassell and 
Meyer were also substantial. The interesting point is that all of the elementary 
grade mini-schools are providing elementary education but the cost per student 
ranges from $577 to $729, a variation of about $150 in the cost of their combination 
of resources. In a district of 12,000 elementary students this could mean an expendi- 
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I nsrrucrionol cost per clossroom (in $ thousonds^ 



Culturol Arts 



Doily Livir>g 



Troditionol Plus 



Kindergorten | 



Activity - Centered 



7th Grode School 



Developmenfol Reoding 



Troditionol 

^ Enrichment 



Continuous Progress 



I ndividuol i zed Leorning 



Basic Skills 



Sullivon BRL 



Fine Art 



School 2000 



jlndividuol Study 



Multl -Culfurol 



Acodemic Skills 



Three R's Plus 



CreuHve Ar^i 



Fine Arts 



Math /Science 



Instructionol cost ( in S ) per student (30 students per clossroom) 



Fig. V-C.3 — Comparable mini-school instructional cost per 
classroom and instructional cost per student 



ture difference of $1.8 million a year. This assumes, of course, that the outcomes of 
the particular combination of resources are acceptable. The differences among the 
mini-schools that are identified as a result of estimating the comparable instruction- 
al cost per classroom or student serve to signal outstanding mini-schools for a more 
detailed investigation of their instructional strategy. 



Appendix VI-A 
COMMUNITY OPINION SURVEY 



This is the form of the survey questionnaire which was administered to voucher 
parents. Other samples received slightly different versions of this questionnaire. 

Field Research Corporation 
14 5 Montgomery Street 
San Francisco, California 



389-002 
092272 



0MB Approval NO: 116-S72014 
Expires December 1, 19 72 



COMMUNITY OPINION SURVEY 



WHITE FORM: TO BE USED WHEN 
INTERVIEWING SPECIFIC PARENTS 
K-8 IN THE DEMONSTRATION AREA 



As you may know, interviews are being made among a random sample of pareats 
who have children in the Alum Rock public schools, and I'd like to speaK to 
(NAME FROM FRONT PAGE) if I may, please. 



IF DESIGNATED PERSON NOT AT HOME, FIND OUT BEST TIME TO CALL. BACK AND 
MAKE AN APPOINTMENT TO RETURN THEM. 



Call back on: 



(day) 



(time) 



CONTINUE WITH PROPER RESPONDENT 

The reason for the survey is to find out what people in this area think of 
the schools here and the kind of education they feel that children in this 
district are receiving. 

There are no "right" or "wrong" answers to any of the questions — we just 
want to know what you think. My company is completely independent from 
the school district, and no one at the schools will ever know what you tell 
me. Your answers will be kept strictly confidential. 



NOW PROCEED V'lTH INTERVIEW. 



Time began J 



AM 
PM 



44 



45 



HAND RESPONDENT CARD A 



\ Becjin Card 12 Here I 

22/ 



1. 



A. 



Child's 
Name 



How many children do you have in these schools? 

IF ONE, SAY: Is that a boy or a girl? RECORD ON 

LINE NO. 1 BELOW. What is (his/her) 
name? RECORD ON LINE NO. I. ASK 
Q. 3 TO 6 BELOW THEN GO TO NEXT PAGE c 

IF MORE THAN ONE, SAY: I would like to talk to you 

about what grades your children are in, 
what schools they go to, what you think 
of those schools and so on. To do this, 
let's start with the oldest child you 
have in school this year, who woul3 that 
be? ASK QUESTIONS 2 THROUGH 6 FOR THAT 
CHILD. THEN SAY: "Now, what about the 
next oldest?" REPEAT UNTIL YOU HAVE 
LISTED ALL OF THE CHILDREN IN SCHOOLS 
SHOWN ON CARD AND ASK QUESTIONS 2 
THROUGH 6 FOR EACH. 



Q* 2 Q. 3 Q . 4 

Record How old What grade 

sex: is he/ is he/she 
she? 



in this 
school year? 



Qj 5 

What school 
does he/she 
go to this 
school year? 



Line 
No. 



1 

14/ 



2 

23/ 
3 

32/ 
4 

41/ 
5 

50/ 
6 

59/ 



M 
1 
1 
1 
1 
1 
1 



A2e_ 



Grade level School name 



15/ 16-17/ 



24/ 25-26/ 



18-19/ 
27-28/ 



33/ 34-35/ 



36-37/ 



42/ 43-44/ 



45-46/ 



51/ 52-53/ 



60/ 61-62/ 



54-55/ 
63-64/ 



7 1 

68/ 



69/ 70-71/ 



72^73/ 



20 



29 



38 



47 



56 



65 



2 2 



30 



39 



48 



66 



74-75 



Ques. 6 1 Did he/she attend a nursery school or 
any of pre-school programs such as 
Head Start? 



12/13 



Q. 6 

TSee 

wording 

below*) 



Yes No 
1 2 



1 
1 
1 
1 
1 
1 



22/ 



31/ 



iO/ 



49/ 



58/ 



67/ 



76/ 



ERLC 



46 



1. B. How many children do you have jn schools not listed 

on the card I just gave you? RECORD NUMBER >> Number: 

IF NONE CIRCLE "0" AND SKIP TO PAGE 4 ^ 

IF ONE SAY: Is that a boy or a girl? RECORD ON LINE 
NO. 1 BELOW. What is vhis/her; name? 
RECORD ON LINE NO, 1. ASK Q. 3 TO 6 
BELOW THEN GO TO PAGE 4. 

IF MORE THAN ONE, REPEAT IF NECESSARY: I WOUld like to 

talk to you about what grades your children 
are what schools they go to, what you 

think of those schools and so on. To do 
this, it helps if we can refer to them by 
name. Let's start with the old est of these-'" 



THEN SAY : Who would that be? ASK QUESTIONS 
2 THROUGH 6 FOR THAT CHILD. THEN SAY: 
"Now, what about the next oldest?" REPEAT 
UNTIL YOU HAVE LISTED ALL OF THE CHILDREN IN 
SCHOOLS NOT ON CARD AND ASK QUESTIONS 2 
THROUGH 6 FOR EACH. 







Q. 


2 


Q. 3 


Q. 4 


Q. 5 






Record 


How old 


What grade 


What school 






sex: 


is he/ 


IS he/she 


does he/she 










she? 


in this 


go to this 












school year? 


school year? 


Child's 


Line 












Name 


No. 


M 


F 


Age 


Grade level 


School name 




1 


1 


2 










14/ 




15/ 


16-17/ 


IS -19/ 






2 


1 


2 










23/ 




24/ 


25-26/ 


27^28/ 






3 


1 


2 










32/ 




33/ 


34-35/ 


36-37/ 






4 . 


1 


2 










41/ 




42/ 


43-44/ 


45-46/ 






5 


1 


2 










50/ 




51/ 


52-53/ 


54-55/ 






6 


1 


2 










59/ 




60/ 


61-62/ 


63-64/ 






7 


1 


2 










6 8/ 




69/ 


70-71/ 


72-75/ 





29 



38 



47 



56 



65 



30 



3 



39 



48 



57 



66 



74^75 



Ques. 6: Did he/she attend a nursery school or any 
of pre-school programs such as Head Start? 



Q. 6 

(See 
wording 
below*) 



Yes No 



1 
1 
1 
1 
1 
1 
1 



2 2/ 



31/ 



40/ 



49/ 



58/ 



67/ 



76/ 



ERLC 



47 



\Begiy) Card 14 Here'^ 



Now I'd like to talk to you about, how your (child is/ohildren are) getting 
along m school. We are especially interested ix\ chiJdren in grades from 
kindergarten through 8th» 



INTERVIEWER: 

RECORD NUMBER OF CHILDREN FROM Q. 
ON PAGE 2 HERE: 



IF ONE CHILD, SKIP TO NEXT PAGE 

IF MORE THAN ONE, CONTINUE AS FOLLOWS: 

I have a chart here which tells me which of your children an grades K-8 that 



go to public schools I should like to ask about first, 
numbers out of a hat • 



It's like pulling 



NOW LOOK ACROSS~THE CHART EltOW AND FIND THE NUMBeF'"' 
OF CHILDREN THAT YOU JUST RECORDED ABOVE FROM Q.1,A= 
ON PAGE 2. UNDERNEATH IT YOU WILL FIND THE LINE 
NUMBER ON PAGE 2 OF THE CHILD ABOUT WHO" VOU WILL 
ASK Q, 7-24 



LABEL #2 



12-18 



RECORD LINE NO. HERE OF SELECTED CHILD 

LOOK ON PAGE 2 FOR THE SELECTED CHILD'S 
NAME SO YOU CAN USE IT IN Q, 7--24 
RECORD NAME OF CHILD HEP£ 



T97 
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I Card 14 \ 

Ml right, let's start with (NAME OF CHILD). 



7. Do you think (CHILD) likes school this year most of the time, some of 
the time, or almost none of the time? 

Most of the time 1 20/ 

Some of the time 2 

Almost none of the time . . 3 

Don't know 0 

Let's see, (CHILD) goes to (NAME OF SCHOOL). (INTERVIEWER: LOOK ON PAGE 2 
FOR NAME Ck SCHOOL THIS CHILD ATTENDS.) 

8, In general, do you think that the education (CHILD) is getting at 
(SCHOOL) is very good, good, fair, or poor? 

Very good 1 21/ 

Good 2 

Fair 3 

Poor 4 

Don^t know . . = 0 



9, Do you think the principal is doing a very good job, a good job, a 
fair job, or a poor job? 

Very good 1 22/ 

Good 2 

Fair 3 

Poor 4 

Don't know 0 

10. Do you think his/her teacher is doing a very good job, a good job, a 
fair job, or a very poor job? 

Very good 1 23/ 

Good 2 

Fair 3 

Poor 4 

Don't know 0 

11, Does (CHILD) like (his/her) (homeroom) teacher a lot, somewhat, a 
little, or not at all? 

A lot 1 24/ 

Somewhat 2 

A little 3 

Not at all 4 

Don't know 0 

O 

ERLC 



49 



17, 



So far this school year, how do you thjnk (CHTLD) as doing m (his/her) 

school work? Is (he/she) doing very well, fairly well, not too weH, 
or very poorly? 

Very well 1 25/ 

Fairly well 2 

Not too well 3 

Very poorly 4 

Don't know 0 

Has (CHILD) ever gone to another elementary school besides the one 
(he/she) attends now? 

Yes (ASK A AND B) 1 55/ 

No 2 

IF YES : 

A. How many other schools has (he/she) gone 

to in total? (No, of other 

schools) 

B. When did (CHILD) start going to the school he attends now — 
this school year, during the last school year (1971-72) , or 
before that? 

This school year 1 ^^/ 

During last school year . . 2 
Before that 3 



3 



18. 



How many years of school would you personally like (CHILD; to 
complete? READ CATEGORIES AND CODE IN COL. A. 



How many years of school do you think (CHILD) will actuall y 
complete? READ CATEGORIES AND CODE IN COL. B. 



Some college, technical, business 
or trade school ...... 



Post graduate or professional school 



I Do not read } - 





B 1 


3?/ 
1 


38/ 
1 


2 


2 


3 


3 


4 


A 

1 


5 


5 


0 


0 
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I Ca)'d 17 } 

QUiEZJ 

23. Everything considered, how satisfied are you with how (CIIlhD; is doang 
in school these days--very satisfied, somewhat satjsfied, somewhat 
dissatisfied, very dissatisfied? 

Very sat i si led 1 4 6/ 

Somewhat satisfied , , , . ? 
Somewhat dissatisfied . . 'i 
Very dissatisfied 



24. Before (CHILD) entered school about how often would you say that you 
or members of your family read to him--f requently , occasionally, 
or hardly ever? 

Frequently 1 4 4/ 

Occasionally 2 

Hardly ever 3 



HAND 
CARD B 



25. Now we would like to know hov; satisfied you are with various groups 
and people connected with the schools. Using the categories on this 
card, please tell me how satisfied are you with the job (CHII«D*S) 
(homeroom) teacher is doing? (RECORD BELOW) 

26. How about the principal how satisfied are you with the job the 
principal of (CHILD'S SCHOOL) is doing? (RECORD BELOWJ 

27. How about Parent's Associations or the PTA — how satisfied are you 
with the jDb they are doing in (CHILD'S SCHOOL)? (RfXORD BELOWJ 



28. In general, how satisfied are you with the job the Alum Rock Board 
of Trustees (School Board) is doing? (RECORD BELOW) 



(CHILD'S) (homeroom) 
teacher 


Very 

satisfied 


Somewhat 
satisfied 


SbmewhaT^ 
dissatis- 
fied 


^Very^ 
dissatis- 
fied 


No j 
opinion [ 

\ 

I 

0 1 

1 


1 


2 


3 


4 


Principal of 
(CHILD'S SCHOOL) 


1 


2 


3 


4 


0 i 
[ 


Parent's Associatior 
(PTA) 


1 




2 


3 


4 


0 i 

„ 1 


Alum Rock Board 
of Trustees 


- 1 


2 

J 


3 


4 


i 



ERIC 
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I Card if 



29, 



A. 



I will now read statements often made by people about government, 
schools, and current issues. Would you tell me how strongly you 
agree or disagree with each of these statements by selecting one of 
the answers on this card? 

I Strongly 
Lajree 



I HAND 

[ card C 



Agree | 



Don 
know 



i'tj Di 



s 
ree 



j Strongly I 
Jj)isa2££f J 



Parents should have more to 
say about v/hat their 
children learn in school 1 . 



^9/ 



B. 


The really important deci- 
sions in a household should 
be made by the man 


1 


2 


3 


4 


5 


C. 


I care a lot what my 
neighbors think 


1 


2 


3 


4 


5 


D. 


Parochial or church related 
schools should receive financial 
aid from public tax money 1 


2 


3 


4 


5 


E. 


Nowadays most politicians 
care too much about the 
poor and not enough about 
the average man 


1 


2 


3 


4 


5 


F. 


Obedience and respect for 
authority are the most 
important things children 
can learn 


1 


2 


3 


4 


5 


G. 


In general teachers and 
principals don't want the 
advice of parents 


1 


2 


3 


4 


5 


H. 


Most parents like the idea 
that they should have a 
choice about the kinds of 
schools their children 
attend 


1 


2 


3 


4 


5 


I. 


Children will get a better 
education if their parents 
can select the school that 
they go to 


1 


2 


3 


4 


5 


J. 


It IS the duty of the 
government in Washington 
to provide a 30b for every- 
body who wants to work 


1 


2 


3 


4 


5 




Busing elementary school 
children to schools in other 
parts of the city only 
harms their education 


1 


2 


3 


4 


5 



52/ 



53/ 



54/ 



55/ 



:RJC 



52 



I Begin Cax^cl 15 Here \ 



In general, how satisfied are you with the kind of education your 
child (ren) can get in Alum Rock — are you very satisfied, somewhat 
satisfied, somewhat dissatisfied or very dissatisfied? 



2 1/ 



Very satisfied ^ASK A) 1 

Somewhat satisfied (ASK A) 2 

Somewhat dissatisfied (ASK B & C ON NEXT PAGE) . 3 

Very dissatisfied (ASK B & C ON NEXT PAGE) ... 4 

IF VERY OR SOMEWHAT SATISFIED ; 

A. Even though you are generally satisfied with the kind of education 
your child (ren) can get in Alum Rock, are there some changes that 
you think the schools should make which would give your child (ren) 
an even better education? 



Yes (ASK (1) AND (2) ) . . 1 

No 2 

Don ' t know 0 



12/ 



IF YES TO A ; 

(1) What kinds of changes? 







13-14 








1 


1 



(2) Have you or anyone you know tried to do anything to get the , 
schools to make those kinds of changes? 

Yes (ASK (3) ) 1 19/ 

No 2 

(3) IF YES TO 2; What have you (they) tried to do? 



20-2} 



53 



Card 

(continued) 
IF VERY OR SOMEWHAT DISSATISFIED ; 

B, What are some of the things you would like to see changed 

or things you think would give your child a better education? 





1 


24*25 






26^27 







28*29 



Have you or anyone you know tried to do anything about 
these things? 

Yes (ASK (1) ) 1 30/ 

No (ASK (2) ) 2 

IF YES TO C < 

(1) What have you (they) tried to do? 







31-32 







IF NO TO C < 

(2) What, if anything, do you think you could do about it? 



3S-36 



54 



A. Now let's see... your chiic3(rrn) fis/.'ir.-') in COM': j-' » '^v , A v.n.OW TUh: 
SCHOOL RF.SPONDFMT'S CU II.D { P.F'.N ) ATTllNIX^'). (RfFKi* vr> 0. ^, PAOF 2.) 

B. Here is a list of soinf of the eler.ic?nt ary a.jd midf^jc r,r }ioo.lfi in Alum Rock » 
{HAND CARD A) Going down that list, please tell Pi wfiicb ones you havr 
ever heard of. CODE IN CO!., h ALL PFPPONDENT HA.*-"^ imARD ov. 

Can you tell me the names of any otl-er puljlic eli vrr.tT^yy oy middle schools 
in Alum Rock. CODE IN COL. B ALL SCHOOLS RESPONDFNT NAMF.S. 

C. FOR EACH SCHOOL CIRCLED IN EI THER COL . A OP. B. Do vo^^ Vhow sone thine; abCut the 
quality of that school or not? CODf: TiNDER COLUVK 

D. Now we'd like to get a little better idea of what you thi nk of the schr--. 
your child{ren) go(es) to. Here is a picture cf a ladder (SKOW CARD U) . 
Suppose we say that the top of the ladder (POINTING) is the very best school 
you can think of and the bottoro (POINTING) is the very worst-, school yoa can 
think of. Where on the ladder (MOVING FINGER rapidly UP ANI.' DOWN) would you 
place (FIRST SCHOOL CODED IN COL. A)? WRITL iN STKP NtJMBER IN COL. D. 
REPEAT FOR EACH SCHOOL CODED TN COL. A. 

Now, let's see, you said you knew something about. 'hiC' quality of (FIRST 
SCHOOL CODED UNDER "SOMETHING" COLUMN). Based on what you know about that 
school, where on the ladder would you place it? ViPlTF. IN STEP NUMBER IN COL. 
D. REPEAT FOR EACH SCHOOL KNOW "SOMETHING" ABOUT. 



A. B. r. D. 

Knov.^lt^ dg e 

School Others 

child (rer) have Son-e- Step 

attend hear d of thing Not hing Number 



CASSELL 1 2 ^.t/ _3 4 11/ 35/ 

GOSS l" S <0/ 3 33 22/ 36/ 

HCCdLLM ~" 1 47/ "^I^ ^2/ 37/ 

MEYER 1 '^STZ^ .-4- 

MILLER "I "5 4 3/ "3 " ' ""4 15/ 39/ 

PALA HH y— ^ 26/ 4 0/ 



ARBUCKLK 1 2 45/ 3 A 27/ 41/ 

CONNIFF 1 2 ^6/ T ^ 28/ 42/ 

CURETON 1 2 47/ '\ 4 19/ 43/ 

d6rSA 1 T" 48/ 3 4 20/ 44/ 



FISCHER 1 2 49/ 3 4 22/ 45/ 

GEORGE 1 " 2 50/ ^ 4 22/ 46/ 

HUBBARD 1 2 52/ 3 23/ 47/ 

LINDA ViS'l'A 1 ^ 52/ 3 4 24/ 48/ 



LYND/vLE 1 2 53/ 4 25/ 49/ 

MATHSON l" T' 54/ 3 C 2^/ 50/ 

MAYFAIR 1 _ " g 55/ ^ _ , 27/ 52/ 

PAlN'tER 1 2 56/ 31 ZILZ 25/ 52/ 



ROGERS 1 2 57/ 3 4 29/ 53/ 

rVAN 1 58/ I 4 30/ 54/ 

Th^ h^^OmO I 2 59/ H 4 32/ 55/ 

SHEPPARD I T 60/ 3 4_ 32/ 56/ 



SHIELDS 1 2 ei/ 3 4 33/ 57/ 

SlONAKER ' 1 ^ ^2/ j 4 34/ 56/ 



ERLC 
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] Begin CQ7->d 17 HeY*T^ 

Now I'd like to talk more about the school (s) your child (ren) in grades 
kindergarten through eighth grade go(es) to... 

Let's see, your child (ren) attend (s) . . . WRITE IN NAMKS OF SCHOOLS 
(GRADES K-S) 

NAMES OF SCHOOLS CHILD (REN) GO(ES) TO: 



(write in) (write in) (write in) 
ASK FOR EACH SCHOOL: j j j . |. | ^ ^ ^ 

11-12 20-21 29-30 

32. Do you personally think that the number of children in (SCHOOL; from 
each of the racial or cultural groups is about right, or do you think 
there should be fewer of some, or more of some, or what? (CODE ALL 
THAT APPLY.) 



Would 
like: 



a. 


About right 


1 


13/ 


1 22/ 


i 


31/ 


b. 


Fewer chicano 


2 


14/ 


2 23/ 


2 


32/ 


9i 


Fewer Anqlo (white) 


3 


^5/ 


3 24/ 


3 


33/ 


d. 


Fewer black 


4 


16/ 


4 25/ 


4 


34/ 


e. 


More chicano 


5 


17/ 


5 26/ 


5 


33/ 


f . 


More Anglo (white) 


6 


18/ 


6 27/ 


6 


36/ 



More black 7 19/ 7 28/ 1 37/ 



33. As you know, a certain amount of money goes to the public elementary 
schools here in Alum Rock. Do you think the schools are getting too 
much money, not enough money, or about the right amount to do their job^ 

Too much 1 ^8/ 

Not enough 2 

About right amount ... 3 

Don't know 0 



34, Are the schools making very good, 
money they get, as far as you can 



good, fair, or poor use of the 



tell? 

Very good use . . . . , 1 39/ 

Good use 2 

Fair use 3 

Poor use 4 

Don't know 0 
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I Ca rd 7T1 



35. I will now read some more statements and you tell me how strorqly you 
agree or disagree with each of these statements by selecting one of 
the answers on' this card. 





CARD C 1 


Strongly 
Agree 


Agree 


Don't 
Know 


Dis- 
agree 


Strongly 
Disagree 




A. 


Getting ahead depends on 
who you know more than how 
well you do something 


1 


2 


3 


4 


5 


40/ 


B. 


Elected officials in 
general can be trusted 


1 


2 


3 


4 


5 


41/ 


C. 


In some cases it's best 
for children to attend 
elementary schools outside 
their own neighborhood 


1 


2 


3 


4 


5 


42/ 


D. 


Giving parents a choice about 
the schools their children 
attend will make teachers 
more responsive to their 
complaints and suggestions 1 


2 


3 


4 


5 


43/ 


E. 


Political corruption is a 
necessary evil in govern- 
ment 


1 


2 


3 


4 


5 


44/ 


F. 


In the next few years, 
things are not really going 
to improve for the average 
man 1 


2 


3 


4 


5 


48/ 


G. 


Teachers in the schools of 
Alum Rock should live in 
Alum Rock 


1 


2 


3 


4 


5 


46/ 


H. 


People like roe don't have 
anything to say about what 
the government does 


1 


2 


3 


4 


5 


47/ 


I. 


Over the past few years / 
blacks have got more than 
they deserve 


1 


2 


3 


4 


5 


48/ 


J. 


Public officials don't 
really care what people 
like me think 


1 


2 


3 


4 


5 


49/ 
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Card 17 I 







Yes 


No 


opinion 




A. 


Do you think parents should be able tc 
help decide which teachers qet hired or 
fired m their children's schools? 


1 


z 


U 


ou / 


B. 


Should parents be able to help decide 
whether a principal is hired or kept on? 


1 


2 


0 


51/ 


C. 


And should parents be able to help 
decide what should be taught in school? 


1 


2 


0 


5 2/ 


D. 


Should they be able to help decide how 
the school spends its money? 


1 


2 


0 


53/ 









Yes 


No' 




These 


next questions are about voting and politics. 








37. 


A. 


Have you or any other member of your 
family ever worn a campaign button? 


1 


2 


54/ 




B. 


Have you or any other member of your 
family ever worked for a political 
candidate? 


1 


2 


56/ 




C. 


Are you registered to vote in this area? 


1 


2 


56/ 



And now on a somewhat different subject. 

38. Have you heard about the educational voucher program that started m 
some of the schools here in Alum Rock this year? 

Yes 1 57/ 

No (SKIP TO Q. 53) » 2 



39. How good a job do you feel the school system has done in explaining 
the voucher system and how it works--do you think they have done a 
very good job, a good job, a fair job, or a poor job? 

A very good job ... 1 58/ 

A good job 2 

A fair job 3 

A poor job 4 

No opinion 0 
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I Begin Card IS Hen^' l 



40. Now we would like to find out where and how much information you 

received about the voucher system and the different programs being 
offered by the schools. 

HAND 
CARD E 



A. First, in which of these ways did you get any information about 
the voucher system and the different programs being offered by 
the schools? (RECORD UNDER COLUMN A) . 

B. Considering all the different ways you learned about the voucher 
system, which was the most helpful to you? (CODE IN COL. B) . 

C. Which was the least helpful to you? (CODE IN COL. C) 



a. From materials received from the schools 


A 


B 


C 


1 


33-34/ 
1 


35-36/ 
1 


b. From talking to children 


2 2 3' 14 


2 


2 


c. From attending parent meetings 


3 16-16 


3 


3 


d. From meeting personally with counselors and 
advisors from the schools 


4 17^18 


4 


4 


e. From attending School Board meetings 


5 19-20 


5 


5 


f. By reading about it in newspapers 


6 27-22 


6 


6 


g. From hearing about it on radio and TV 


7 23-24 


7 


7 


h. From attending PTA meetings where it 
was discussed 


8 25-26 


8 


8 


i. From bulletins or newsletters from the school 


9 27-28 


9 


9 


j. By talking to other parents, relatives, 
friends, or neighbors about it 


10 29-30 


10 


10 


k. By talking to teachers or principals about it 


11 31'-32 


11 


11 
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"Sard 18 I 

41, A. The school district gave parents enrollment forms to fill out for 
their children. Did you personally fjll out these forms? 

Yes . . (CODE "1" IN COL. A BELOW) 

No . . Ask: Who did? (CODE IN COL. A) 

B. Did (YOU/PERSON CODED IN COL. A) talk with anyone else about which 
program and which school to choose for your children? 

Yes . . ASK: Who was that? (CODE IN COL. B 

BELOW ALL THAT APPLY) 

No . . (CIRCLE CODE "9" IN COL. B) 





A 












Respondent 


1 Z?/ 


1 


38/ 


(ASK 


C 


& D BELOW; 


Respondent ' s (husband/wife) 


2 


2 


Z9/ 








Other adult in family 


3 


3 


40/ 








Child/children themselves j 


4 


4 


41/ 








Other children outside the family 


5 


5 


42/ 








Teachers 


6 


6 


4Zf 


(ASK 


E 


ON NEXT PAGE; 


Parent Counselors or advisors 
at school 


7 


7 


44/ 


(ASK 


F 


ON NEXT PAGKj 


Other parents, friends, neighbors 


8 


8 


45^ 


(ASK 


G 


ON NEXT PAGf:; 


No one helped me 




9 


46/ 








Don't remember 


0 


0 


4-^/ 









IF RESPONDENT MADE CHOICE OR HELPED TO MAKE CHOICE, ASK C AND D BELOW : 
^"^ When it came time to make a choice, did you find you had enough 
information about the different programs being Oi-tered in the 
school (s) you selected for your child (ren) or would you have liked 
to have had more information about the programs offered there? 

Had enough information . • 1 4 8/ 
Wanted more information . . 2 
Can't say 0 

D. What about the other schools in the voucher system, did you feel you 
had enough information about the different programs they were 
offering or would you have liked to have had more information about, 
the programs offered at the other schools? 

Had enough information . . 1 4 9/ 
Wanted more information . . 2 
Can*t say 0 
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IF TEACHER MADE CHOICE OR HELPED: 
E. How good a job do you feel 

teachers have done in explain- 
ing the voucher system and how 
it v;crks — do you think they have 
done a very good job, a good 
30b, a fair job, or a poor job? 


Very 
good 
job 


Good 
job 


Fair 
30b 


Poor 
30b 


1 


2 


3 


4 


IF PARENT COUNSELLORS/ ADVISORS MADE 
CHOICE OR HELPED: 

F. How good a job do you feel parent 
counselors have done in explain- 
ing the voucher system and how it 

done a very good job, a good job, 
a fair job, or a poor job? 


1 


2 


3 


4 


IF FRIENDS, NEIGHBORS, OTHER PARENTS 

MADE CHOICE OR HELPED: 

G. How good a job do you feel the 

friends, neighbors or parents you 
talked with have done in explaininq 
the voucher system and how it 
works — do you think they have 
done a very good job, a good job, 
a fair 30b, or a poor job? 




2 


3 


4 
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Card 18 



42. Here is a list of groups and people that the voucher systt:m will affect. 



HAND 
CARD F 



ASK ABOUT EACH GROUP LISTED BELOW: Do you think that (GROTJP) will be 
helped or hurt by the voucher system? (RECORD RESPONSE UNDER COL. A. 
THEN ASK: Do you think they will be (helped/hurt) a lot or a little? 
(CODE UNDER COL. B.) 











Col 


. A 






Col. B 








Helped 


Hurt 


Neither 


Don* t 
know 


A 

Lot 


A 

Little 




A. 


Children from lower 
income families 


1 


2 


3 


4 


53/ 


1 


2 


61/ 


B. 


Children from middle 
and upper income 
families 


1 


2 


3 


4 


54/ 


1 


2 


62/ 


C. 


Black children 




1 


2 


3 


4 


55/ 


1 


2 


63/ 


D. 


White/Anglo children 


1 


2 


3 


4 


56/ 


1 


2 


64/ 


E. 


Chicano or Mexican 
American children 




1 


2 


3 


4 


5V 


1 


2 


65/ 


F. 


Teachers 




1 


2 


3 


4 


38/ 


1 


2 


66/ 


G. 


School administrators 


1 


2 


3 


4 


69/ 


1 


2 


67/ 


H. 


Parents 




1 


2 


3 


4 


60/ 


1 


2 


68/ 



As you may know, under the voucher system, each school offers several different 
ways of teaching children. These are called programs. The idea is for parents 
to be able to select from different types of educational programs iznd classroom 
approaches the one they think would be best for their children. 



43^ Taking everything together, do you think giving parents a choice between 
different types of programs is a very good idea, a good idea, a fair 
idea, or a poor idea? 

Very good idea ... 1 e9/ 
Good idea ..... 2 

Fair idea 3 

Poor idea 4 

No opinion 0 



ERIC 



62 



44. This school year, would you say that there are too many different 
programs availablei too few, or just about the right number? 

Too many 1 '^0/ 

Too few 2 

About the right number . . 3 

No opinion 0 



45. At the time , you had to decide which school your child (ren) would 

attend, was there some kind of special program you wanted the schools 
to offer that they didn't offer? 



Yes (ASK A) 
No 



1 
2 



A. IF YES: What was it you wanted that they didn't offer? 



72/ 



46. How satisfied are you with the kinds of programs offered this school 
year — very satisfied^ somewhat satisfied, somewhat dissaticf led, or 
very dissatisfied? 

Very satisfied I 78/ 

Somewhat satisfied . . • . 2 
Somewhat dissatisfied . • 3 
Very dissatisfied .... 4 
No opinion 0 
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[ Becjin Card 19 Here \ 

47. Overall, do you think the chances of your child (ren) getting the kind 
of education you want for (him/her/them) are better under the voucher 
system, about the same, or not as good as they were last year? 

Better 1 12/ 

Same • • 2 

Not as good 3 

No opinion 0 



48, As far as you know, under the voucher system, if a parent wants to send 
his child to a school that is not in his own neighborhood, does the 
parent have to provide transportation himself, does the child use city 
buses, or is transportation provided free of charge? 



Parent must provide transportation .1 12/ 

Child uses city buses 2 

Transportation free of charge ... 3 
Don ' t know 0 



The next few questions are about the school (CHILD ABOUT WHOM Q. 7-24 WERE 
ASKED) goes to. (INTERVIEWER: REFER BACK TO PAGE 4 FOR LINE NUMBER AND NAME 
OF CHILD.) 

49. Why did you select that school for (CHILD)? 



1 


12' 






15' 


'16 


1 


1 



17-28 



50. Is (CHILD) attending the "school" nearest your home, or is (he/she) 
going to a different school? 



School nearest home 1 19/ 

Different school 2 



51. Some schools have more than one program. Do you happen to know which 



program (CHILD) is enrolled in? 

Yes (ASK A AND B) 1 20/ 

No , 2 

IF YES; _____ 

A. What is that program called; 

B. Is this your first choice for (him/her)? 22 22 22 

Yes 1 2^/ 

No 2 

Don't remember 3 
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[ _C j3 r d " : 9 

52. Is there anything about the particular program (CHILD) is in that you 
think might be. especially good or especially bad for (him/her )f 



Yes, especially good (ASK A) . . 1 

Yes, especially bad (ASK B) . . 2 

Yes, both (ASK A AND B) .... 3 

No, neither 4 



28/ 



A. IF ESPECIALLY GOOD OR BOTH: 



B. IF ESPECIALLY BAD OR BOTH: 



What is it about the prograna that 
you think will be especially good 
for (CHILD) ? 



n 



28-29/ 



What IS it about the program that 
you think will be especially bad 
for (CHILD)? 



m 



32-^3/ 



53. In your experience does (CHILD'S) (homeroom) teacher pay a lot, sorr 
or hardly any attention to suggestions or complaints from parents? 

A lot 1 

Some 2 

Hardly any 3 

Don't know 0 



2^/ 



54. How about the principal — does the principal pay a lot of attention, 
some, or hardly any attention to suggestions or complaints from 
parents? 

A lot 1 3S/ 

Some 2 

Hardly any 3 

Don't know 0 
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55. 


Now I hc:ve i: t't'W statements about the voucher system and I w»3uld like 
you to tell njo how strongly you agree or disagree wath each of them by 
selecting one oL the an?;wers on this card. 




Fhand 

1 CAKD C 1 


Strongly 
Agree 


Agree 


Don't 1 

Know 1 Disagree 


StronoTv 1 
Disagree J 




A. 


The voucher system will 
result in better public 
schools .in Alur^ Rock 




1 


2 


3 4 


5 


36/ 


B. 


Under the voucher system 
a p^ient c^n change the 
schcol his child attends 
at regular tints during 
the school year 


1 


2 


3 4 


5 


?/ 


C. 


Some families will leave 
Alum Rock rather than have 
their children go to 
voucher schools 


1 


2 


3 4 


5 


38/ 


D. 


Parents should be able to 
use vouchers in parochial 
and private as well as 
public schools 


1 


2 


3 4 


5 


39/ 


E. 


Under the voucher system 
the more schools a parent 
has to choose from, the 
r.'ore control parents will 
have ever the school 
system 


1 


2 


3 4 


5 


40/ 


F. 


The voucher system will 
do more for the education 
of blacks and Chicanos than 
it wjII for the education 
of v/hites 


1 


2 


3 4 


5 


'Si/ 


G. 


Under the voucher system 
some ft Chen Is would have too 
many students and other 
schools would have empty 
class: ccms 


1 


2 


3 4 


5 




H. 


For most pr^rents how close 
e[ school is to home is the 
rrost important reason for 
choosing ct school for his 
children to attend 


1 


2 


3 4 


5 


43/ 


I. 


The voucher system will 
give parents more control 
over the schools of 
/aurr. Rock 


1 


2 


3 4 


5 


44/ 
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Now try to thank back to last year before there was any discussion of the 
voucher system. 

56, Did you have any children in school in Alum Rock last year? 

Yes 1 4h/ 

No (SKIP TO Q. 59) 2 



57, How many times did you or other members of your family do any of the 
following things during the last school year . . . (READ LIST BELOW) 
WRITE IN NUMBER OF TIMES OR CIRCLE "0" FOR NONE 

Number 

times None 

A. Have talks with the teachers or 
other people at the school about 

your children 0 46-47/ 



B, Attend any parent or neighborhood 
meetings at which the schools 

were discussed 0 4 8-4 9/ 



Attend any special events at 

elementary schools, such as plays 

or ballgames or special assemblies, etc. 



58^ Last year, did you or any other members of your family belong to any 
cor^nittees or groups at your elementary school^ such as the PTA, 
the Chicano Parents of Alum Rock, or other parent groups? 



Yes (ASK A) 1 52/ 

No 2 

A. IF YES: Which committee or group did you or anyone in your 
family belong to: 

The Chicano Parents of Alum Rock 1 5^/ 

PTA 2 

Other (s) (SPECIFY)_,_- . — . — . — . 3 



0 \->0'Sl/ 
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I Card 2 0 \ 

59^ Last year, did you or anyone else in your family attend any special 
public or neighborhood meetings about local issues not related to 
the schools? 

Yes (ASK A) 1 54/ 

No 2 

A. IF YES ; How many such meetings did you or anyone of your 
family attend? RECORD BELOW 

^ 65-65. 

(total number of public or 
neighborhood meetings) 

60. Does your family have a dictionary here at home? 

Yes 1 57/ 

No 2 



61, Does your family have an encyclopedia here at home? 

Yes 1 58/ 

No 2 



62. Do you receive any magazines regularly here at home? 



Yes 
No 



1 
2 



69/ 



63, About how often do you get a newspaper here at home — every day, a few 
times a week, once a week, or less often than that? 



Every day 1 

Few times a week • , . . 2 

Once a week 3 

Less often 4 

Not at all 5 



6C^ 



And now a few questions about your background. 
64, A. How long have you lived in Santa Clara County? CODE IN COLUMN A. 



Less than 1 year 

One to less than 2 years . 
Two to less than 3 years . 
Three to less than 5 years 
Five to less than 10 years 
Ten years or longer • . . 



62/ 



B 



62/ 



How long have you lived at this address: CODE IN COL. B ABOVE, 



68 



65, 



I Card LJ \ 

Which of these phrases (HAND CARD G) best describes how likely it is 
that you and your family might move away from Alum Rock withjn the 
next two years? 



HAND 
CARD 
G 



Definitely will move 1 

Probably will move 2 

Might move (50/50 chance; ....... 3 

Probably will not move 4 

Very unlikely we will move* 5 



6d/ 



66. Alto '^er, i eluding yourself, how many adults 18 and over live here 
in i*ij.s household most of the year? 

• 64-65/ 

(No. adults in household) 



67, 
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Do you have any children who are not yet in school, that is, tco 
young to be in school? IF NO, CIRCLE "0". IF YES: How many? 



Number not yet in school; 
None 



66-6''/ 



Do you have any children who are out of high school now — just 
count those who live here and who are under 18? IF NO, 
CIRCLE "0". IF YES: How many? 

Number out of school: 

None . 0 



68. Are you working full time now, working part time, (keeping house) , 
(retired), or can't you find work, or what? 

Working full time (35 hours or more) . . 1 

Working part time 2 

Temporarily laid off 3 

Can't find work 4 

On strike 5 

Retired 6 

Keeping house (SKIP TO Q. 70) 7 

Other (SPECIFY) . . 8 



6i. A. What kind of work (do/did) you normally do? 

OCCUPATION 

B. In what industry is this? 
INDUSTRY 



68-69/ 



?1 -^2 73 74 



73 



76 77 



70/ 



69 



I Bec^in Card 20' lieve \ 

70. What is your present age: WRITE IN: 

IF RESPONDENT REFUSES, PLEASE WRITE IN ESTIMATE ABOVE AND CHECK BOX 



71. What was the highest grade or class you completed in school? 



None to three grr Jes I 15/ 

Four to seven gracas 2 

Eight grades 3 

Nine to eleven grades 4 

High school graduate 5 

Junior college graduate 6 

Technical , trade , business p and 

nursing school 7 

Some college, but not a graduate , . • 8 

College graduate or more 9 



72. Are you married, single, divorced, separated, or widowed? 

Married (ASK Q. 73) i 1^/ 

Single (SKIP TO Q. 78 ) 2 

Divorced (ASK Q. 74) 3 

Separated (ASK Q. 74 ) 4 

Widowed (ASK Q. 74) 5 



73. Is your (husband) (wife) working full time now, working part +:ime, 
(retired), or can't (he/she) find work, or what? 

Working full time (35 hours or more) . 1 IZ/ 



Working part time . . * 2 

Temporarily laid off 3 

Can't find work 4 

On strike 5 

Retired 6 

Keeping house (SKIP TO Q.75 ) .... 7 

Other (SPECIFY) . 8 
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74. A. What kind of work (does/didj your ^nusbandj (wife; normally do? 



OCCUPATION ; j 1 I j 

B. In what industry (is/wasj this? 





INDUSTRY 


1 

( 






?0 2 J 


22 


75. 


What is your (husband's) (wife's) age? WRITE IN: 


2S- 9<} 





IF RESPONDENT REFUSES, PLEASE WRITE IN ESTIMATE ABOVE AND CHECK BOX 



□ 



76. What was the highest grade or class your (husband) (wife) completed 
in school? 



None to three grades 1 

Four to seven grades 2 

Eight grades 3 

Nine to eleven grades 4 

High school graduate 5 

Junior college graduate ^ 

Technical, trade, business, and 

nursing school 7 

Some college, but not a graduate . . 8 

College graduate or more 9 

Don't know t .... 0 



IF INTERVIEWING A WOMAN WHO IS CURRENTLY MARRIED, WIDOWED, DIVORCED OR SEPARATED 

ASK Q.77 ABOUT HER HUSBAND'S FATHER. 

IF INTERVIEWING MAN, ASK ABOUT HIS FATHER. 

77. What'kind of work did (your ) (husband ' s) father do most of his life? 



OCCUPAT I ON i I ! ! 

26 2? 28 29 



INDUSTRY : 

3i zt 
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FCard 20 \ 

/8 Which of the words on the card best describes your race or ethnic 
backgrouridV 

j 1i'AND""**1 Mexican Amer ican/Chicano 1 33/ 

L£^^.ILJ Other white/Anglo . 2 

Black/Negro 3 

Oriental (Japanese/Chinese/ 

Filipino/Korean ) 4 

American Indian ,5 

Latin 6 

Other (SPECIFY: ) . 7 

79. What is you-" ^e>.r f,:' ous background? 

Roman Catholic 1 24/ 

Protestant 2 

Jewish 3 

Other (SPECIFY: } . 4 

Refused to answer 5 

80. Do you own your own home here, or do you rent? 

Own 1 3 5/ 

Rent .2 



81 



Would you please look at this card and tell me wh.tch amount comes 
closest to your total fan,ily income from all sources last year, before 
taxes? Include wages, salaries, social security or retirement 
benefits, help from relatives, or public assistance of any kind. 
(Just your best guess.) 



HAND 
CARD I 



A. under $3,500 1 

B. $3,500 - $4,999 2 

C. $5,000 - $7,499 3 

D. $7,500 - $9,999 4 

E. $10,000 - $14,999 5 

F. $15,000 or more 6 

G. Refused 7 

H. Don't know 0 



36/ 



37, Finally, how many people in total depend on this income? Z7-ZQ/ 

Number 

NEXT, FILL IN THE ITEMS ON PAGE 29 



er|c 
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Thank you very much for your help. Please remember that all of the information 
you have given me is strictly confidential. 

PLEASE FILL OUT THE ITEMS BELOW IMMEDIATELY AFTER COMPLETING INTERVIEW 



A. 


Respondent ' s 


Male . . . 


. . 1 


55/ 


C. Was this 


interview conducted in 




sex: 


Female 


. . 2 




Spanish? 




5. 


Respondent' s 


White . . 


. . 1 


40/ 








race : 


Black . . 


. . 2 










Mexican 


American . 


. . 3 












Other . . 


. . 4 









D. How cooperative was respondent? 

Very cooperative . . . . 1 J7/ 
Fairly cooperative ... 2 
Not cooperative .... 3 

E. How interested was respondent? 

Very interested .... 1 22/ 
Fairly interested ... 2 
Not interested .... 3 



F. What is color of form used? 

White 

Pink 

Green 

Blue , 



Yellow 5 



]^ecord interviewer 
number: 



:4 lb 16 17 



I Begin Card 21 Here 



G. Time inter- 




AM 


view ended: 




1 


(hr) 


(min) 


18/ 




19/ 


H. Total length of 






interview: 




1 




(In 


minutes) 



20 



21 



22 



I. Date of 
interview: 



(month) 
23 24 



IHSyl 

26 26 



Quota type met : (CIRCLE ONE) 

Husband under 40 ... 1 

Husband 40 and over . . 2 

Working wife 3 

Non-working wife ... 4 

Male (no wife) ... 5 

Female (no husband) . . 6 



27/ 



Your signature: 



SUJ: 



SUPS 



P.8 P.9 i<0 3i 3Z 33 34 



SURS 
PAGE #: 



3& 38 

.. rn 



39 4 0 



Census 
Tract #: 



Census 
Block #1 



1 4i 43 44 46 4 6 

U 



47 48 49 60 



ERIC 



Appendix VI-B 
PARENT/COMMUNITY SURVEY 
by Robert T, Riley 



INTRODUCTION 

On November 4, 1972, the interviewers of Field Research Corporation (FRO 
entered the field to collect data on 1460 respondents concerning their educational, 
political and voucher-related attitudes and activities. The Alum Rock survey actual- 
ly contains five separate survey instruments, and the Alum Rock sample contains 
six separate subsamples. Parents and nonparents are differentiated from one anoth- 
er as well as respondents who live in the voucher demonstration area and those who 
do not live in the voucher demonstration area. Hereafter we refer to these areas as 
the voucher and nonvoucher areas. In addition, there is a separate subsample of 
changers, parents who changed the school that any one of their children in the 
voucher area attended and a separate probability sample of parents in the non- 
voucher area. 

The parent and nonparent samples were drawn using different techniques: the 
nonparent sample in both the voucher and nonvoucher areas and one of the parent 
samples in the nonvoucher area employed a block quota sampling method, while the 
parent sample in the voucher area, the changer sample, and the special subsample 
of parents in the nonvoucher area employed a probability method. Table VI-B.l 
presents a breakdown of the six samples in terms of the number of respondents by 
the method in which they were drawn. 

For five of these six subsamples the interviewers administered a modification of 
the basic survey form. The voucher parent sample and the changer sample received 
the same survey form. Additionally, there was a Spanish language version, as well 
as an English language version, of each of these forms; thus, in total, 10 separate 
instruments. In Table VI-B.l we also see the number of respondents by subsample 



Table VI-B-1 
SAMPLE TYPE BY LANGUAGE OF INTERVIEW 



Sample Type 


Language 


English 


Spanish 


Total 


Probability samples 








Voucher parents 


508 


92 


600 


Changers 


53 


3 


56 


Nonvoucher parents 


148 


36 


184 


Block quota samples 








Nonvoucher parents 


236 


22 


258 


Nonvoucher nonparents 


215 


27 


242 


Voucher nonparents 


98 


22 


120 


Total » all samples 


1258 


202 


1460 
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that received the Spanish language version of this form. Overall, about one-third of 
all Mexican-American respondents received the Spanish language form. 

Forty interviewers administered the questionnaires throughout the Alum Rock 
area. Approximately one-fourth of these interviewers were bilingual in that they 
were capable of administering the form in either Spanish or English. All but two 
of the interviewers were female; there were two young male interviewers. Moreover, 
there were two black interviewers to interview blacks, who composed a small propor- 
tion of the total interviews. 



COMPLETED INTERVIEWS 

Within the first week of the survey, that is by November 11 (see Table VI-B.2), 
the interviewers completed a total of 564 interviews — 333 from the parent probabili- 
ty sample and 231 from the block quota sample. At the end of the second week, 485 
of the 600 voucher parent interviews and 410 of the 620 block quota interviews were 
complete. This represents over 60 percent of the total interviews; when broken down 
by sample type it reveals that 81 percent of the voucher parent probability inter- 
views and 69 percent of the block quota interviews were complete. 

On November 24, FRC received the names for two additional subsamples of 
interviews to be obtained: one subsample represented parents drawn on a probabili- 
ty basis in the non voucher area from the six schools involved in the RECAP attend- 
ance project: Matheson, Fischer, Hubbard, Shields, Painter, and ConnifT. The other 
subsample represented changers, those parents who changed the school in the 
voucher area that one of their children attended. 

By November 27, 526 of the 600 parent interviews and 510 of the 620 block quota 
interviews were in hand. In addition, nine of the changers and 28 nonvoucher 
probability parents had also been interviewed. The percentage increase in com- 
pleted interviews from the second to the third week decreased because, in general, 
interviews completed at this time represented second, third, fourth, or fifth call- 
backs in the probability sample, and in the block quota sample represented inter- 
views with respondents that had to be located in alternate blocks since the original 
block had been exhausted in the quota search. This decrease does not represent a 
lack of efficiency; it demonstrates the increase in difficulty which faced interviewers 
in obtaining completed interviews. 



Table VI-B.2 

CUMULATIVE NUMBER OF COMPLETED INTERVIEWS 
BY WEEK AND BY SAMPLE TYPE 





Nov . 4- 


Nov. 12- 


Nov. 20- 


Nov. 28- 


Dec. 5- 


Sample 


Nov. 11 


Nov. 19 


Nov. 27 


Dec. 4 


Dec. 12 


Voucher parent proba- 












bility sample 


333 


485 


526 


551 


600 


Changer sample 


0 


0 


9 




56 


Nonvoucher parent 












probability sample 


0 


0 


28 


110 


184 


Block quota sample 


231 


410 


510 


561 


620 


Total 


564 


895 


1073 


126 7 


1460 
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On December 4, at the end of one month, 1267 of the 1460 interviews had been 
completed, which represented over 90 percent of the parent probability sample, over 
90 percent of the block quota sample, 88 percent of Lhe changer sample, and 51 
percent of the nonvoucher parent probability sample. During the final week, from 
December 5 to December 12, interviewers continued to hunt the "hard to find" 
respondents in the probability sample and to search for "hard to locate" respondents 
in the block quota sample. At times in this latter case, interviewers had exhausted 
two, three, or four alternate blocks before they found their last block quota respond- 
ent. 



THE SAMPLES IN DETAIL 

The Probability Sample 

In each of the six voucher schools a sample was selected such that it would 
proportionately represent the total school population and within each school stratifi- 
cation of the sample was such that blacks, Mexican-Americans, and other whites 
were represented in their proportion of the school population. In total, 600 families 
were to be interviewed in all of the voucher schools, and to obtain an 80 percent 
response rate we drew 720 parent names. Table VI-B.3 presents a detailed break- 
down of the parent probability samples. We see that for the parent probability 
sample, 174 of the 600 interviewers were obtained on an initial call and 165 from 



Table VI-B.3 

PROBABILITY SAMPLE TYPE BY INTERVIEW DISPOSITION 





Voucher Probability Sample 




Nonvoucher Probability Sample 






Other 




Mexican- 






Other 




Mexican- 


Item 


Total 


White 


Black 


American 


Changers 


Total 


White 


Black 


American 


Total complete interviews 


600 


244 


56 


300 


56 


184 


62 


13 


104 


First call 


174 


84 


16 


74 


13 


60 


26 


4 


30 


Second call 


165 


54 


13 


98 


16 


36 


12 


4 


20 


Third call 


119 


47 


15 


57 


10 


29 


8 


2 


19 


Fourth call 


57 


23 


3 


31 


6 


26 


5 


1 


17 


Fifth call 


35 


15 


2 


15 


4 


14 


2 


2 


8 


Sixth call 


16 


8 


1 


7 


0 


6 


3 


0 


3 


Seventh call 


18 


5 


4 


9 


3 


6 


3 


0 


3 


Eighth call 


16 


5 


2 


9 


2 


7 


3 


0 


4 


Refusals and terminations 


29 








11 


16 








Language barrier (spoke only 




















Samoan) 


4 








0 


0 








Permanently not at home 


7 








5 


8 








Ciiiiii'iren no longer in school 


4 








1 


1 








Previously interview.^d 


18 








21 


17 








Moved from address 


58 








15 


50 








Total names 


720 


286 


67 


363 


99 


276 








Initial refusals successfully 




















interviewed 


19 








3 


3 
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the first call-back. An additional 119 interviews resulted from a second call-back. 
This represents 468 of the 600 respondents. However, among other white respond- 
ents, interviewing a respondent on the first, second, or third call was proportionately 
higher than among Mexican-Americans; comparisons of the black sample, because 
of its small size, are hard to interpret in these terms. 

We also see that interviewers obtained a total of 29 outright refusals. In fact, 
they initially had 48 outright refusals, but through the effective reassignment of 
initial refusals to other interviewers, FRC reduced this number from 48 to 29. 
Additionally, we see that 58 of the 720 names originally given to FRC for the parent 
probability sample had moved from the address given and could not be located, and 
18 of the parents in the probability sample had been previously interviewed in 
another subsample. The latter usually resulted because their residence address was 
in the nonvoucher areas. If we exclude those who had been previously interviewed 
and those who had moved from their current address, we see that the response rate 
in the parent probability sample is 93 percent. And, without excluding them from 
the sample, we see that the overall response rate is 82,7 percent. Moreover, the 
response rate appears to be constant across ethnic groups, other white, black, or 
Mexican-American. Similar analysis of the changer sample and the nonvoucher 
probability sample are also presented in Table VI-B.3. It is interesting to note that 
in the nonvoucher probability sample the level of geographical mobility of parents 
is exceptionally high. Fifty of the 276 names presented to FRC were no longer living 
at their current address. Looking at the response rates in the changer and nonvouch- 
er probability samples, we see that in both cases more than 80 percent of the usable 
names were interviewed, excluding those who had moved or been previously inter- 
viewed. 

The Block Quota Sample 

The block quota sample, in fact, contains two separate block quota samples. 
Th<*re was a block quota sample of nonparents in the voucher area, and a block quota 
sample of parents and nonparents in the nonvoucher area. In the nonvoucher area, 
respondents were interviewed whether they were a parent or not, such that parents 
fall within the sample in proportion to the overall population. This is to say that we 
did not screen on parents and nonparents. 

The quota types for the block quota sample were as follows: 

1. Males under 40 

2. Males over 40 

3. Working women 

4. Nonworking women 

In addition, as a function of the ethnic composition of the block, either other 
white, Mexican-American or mixed, an ethnic quota was set such that in the mixed 
blocks each of the quota types had an equal probability of being either Mexican- 
American or other white. In the voucher area we see that interviewers obtained 
interviews from 242 nonparents and 258 parents. (See Table VI-B.4.) This reflects 
their distribution in the total population. Comparing the nonvoucher parents and 
the nonvoucher nonparents, we see that the nonvoucher nonparents are dispropor- 
tionately male, 55 percent, and disproportionately older, 54 percent. This results 
from the fact that we did not have an explicit quota on parents, and in fact is 
probably more demographically representative of the population of nonparents and 
parents than equal numbers of each. In the voucher area, the block quota sample 
of nonparents by definition is fixed at one-fourth for each of the quota types. This 
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Table VI-B.4 

BLOCK gUOTA SAMPLE TYPK BY gUOTA TYPE, WITH AND WITHOUT A CONTROL FOR ETHNICITY 





Voucher Nonparents 


Nonvoucher 


Nonparents 


Nonvoucher 


Parents 


Sample 


Total 


Other 
White 


Black 


M-A'"" 


Total 


Other 
Wliite 


Black 


M-A^ 


Total 


Other 
Whire 


Black 


M-A* 


Males under 40 


25.0 


20.0 


16.7 


27.8 


23.6 


21.8 


40.0 


24.1 


26.0 


26.5 


14.3 


26.8 


Males over 40 


25.0 


37.1 


16.7 


20.3 


31.4 


29.8 


20.0 


34.3 


19.8 


22.5 


0 


20.1 


Working women 


25.0 


20.0 


50.0 


25.3 


22.7 


25.0 


20.0 


19.4 


26.7 


22.5 


42.9 


27.4 


Nonworking women 


25.0 


22.9 


16.6 


26.6 


22.3 


23.4 


10.0 


22.2 


27.5 


28.8 


42.9 


25.6 


N - 


120 


35 


6 


79 


242 


124 


10 


108 


258 


80 


14 


164 


Ethnic percentage 
of sample type 




29.0 


5.0 


66,0 




51.2 


4.1 


44.7 




31.0 


5.4 


63.6 



^-A = Mexican-American. 



may include too many younger nonparents in the voucher nonparent sample. But 
this was to be expected because of the sampling method. 

Alternate Blocks. Initially for the block quota samples we had 172 sampling 
units. This number was reduced by 17 when we eliminated those sampling units 
which fell outside the voucher and nonvoucher area geographically and those that 
had too few housing units within their boundaries. In total, 155 sampling units 
remained. These 155 sampling units fell within 118 blocks. The reason for the 
difference between the number of sampling units and blocks is that in some cases 
more than four interviews (between eight and twenty-four) were obtained per block. 
This was to insure that large blocks would be represented in the sample in propor- 
tion to their size. 

One of the major problems with this block quota sample was the inordinate 
number of alternate blocks that it required. In fact, to obtain the required number 
of interviews from the original 118 blocks we had to draw 130 alternate blocks. 
Usually in block quota samples, no more than 20 percent of the original blocks need 
alternates, and in this case the additional 130 blocks brings the total number of 
blocks required for this sample to 248. Given that there were only 280 usable blocks 
in voucher and nonvoucher areas, it became apparent to us that we had saturated 
the area in terms of a block quota sample. Employing this number of blocks out of 
the total severely limited the efficiency with which alternates could be assigned and 
with which interviewers could obtain interviews. A closer analysis of the alternates 
reveals that 86 of the sampling units required at least one alternate. For another 
53 blocks a second alternate was required. For 16 blocks three alternates were 
required and for five bloclts, four or more alternate blocks were required. The large 
number of alternate blocks also helps to explain the extended time it took the 
interviewers to complete many of their block quota assignments. 

From another point of view we see that interviewers needed fewer alternates in 
the other white blocks than in the mixed and Mexican-American blocks. It was also 
hard to assign alternates in the mixed or Mexican-American blocks since the classifi- 
cation of blocks was based on the number of Spanish surnames, not an actual count 
of Mexican- Americans. This may account for the longer period of time it took the 
interviewers to complete interviews in the Mexican-American and mixed blocks. 
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ETHNIC DISTRIBUTION 

The question of how ethnically representative each of the subsamples is remains 
an important one. Table VI-B.5 presents the distribution of respondents in the total 
population of other whites, blacks, and Mexican-American families. In the probabili- 
ty sample we see that 39 percent of the families were other white, 9 percent black, 
and 52 percent Mexican-American. It is important to note that we sampled parents 
in the voucher schools in proportion to the ethnic percentage of the number of 
families in the school population, not the number of students. Using the family as 
the sampling unit rather than the student increases the percentage of other whites, 
since Mexican-American families tend to have more children per family and thus 
overrepresent families in the student population. The parent probability sample 
matches the population from which it was drawn quite closely. In terms of the 
original 720 names drawn, less than one percentage point deviation exists from their 
percentages in the population percentages. In terms of completed interviews, we see 
that other whites are about 1 percent more in the completed interviews and Mexi- 
can-American families are about 1 percent less. 

In the nonvoucher area, the probability sample of parents was drawn on the 
basis of student lists, not family lists. This means that we expected a higher propor- 
tion of Mexican- American families to appear than if we had used a family list. In 
fact, this was the case. The ethnic percentages of nonvoucher parents in the school 
population based on student percentages were: 32.3 percent other white, 10.1 per- 
cent black, and 56.7 percent Mexican -American. The ethnic percentages of the 
sample of completed parents in the nonvoucher area were: 34.6 percent other white, 
7.3 percent black, and 58.1 percent Mexican -American. These slight deviations from 
the population percentages do not appear to bias the sample. 

In some cases the block quota sample is very accurate; in other cases it may be 
less so. In terms of the parent block quota sample in the nonvoucher area, 31 percent 
of the respondents were other white, 5.4 percent black, and 63.6 percent Mexican- 
American. The difference between the 5.4 percent black obtained and the 10.1 
percent expected in terms of the student population probably results from the fact 
that blacks were not a prescribed quota assi.^ment, and given the choice, interview- 
ers sought other respondents. In terms of nonparents in the nonvoucher area, 51.2 



Table VI-B.5 
ETHNIC DISTRIBUTIONS OF SURVEY SAMPLES 







Other 




Mexican- 




Total 


White 


Black 


American 


Sample 


N 


(%) 


U) 


(%) 


Parents from voucher school family Use 


2153 


39.0 


9.0 


52.2 


Parent probability sample 


720 


39.8 


9.1 


51.1 


Completed parent interviews 


600 


40.4 


9.4 


50.2 


Changer sample 


54 


49.1 


11.8 


39.1 


Voucher parent block quota sample 


120 


29.2 


5.0 


65.8 


Parents from selected nonvoucher school 










student list 


4053 


32.2 


10.1 


56.7 


Parent probability sample 


184 


34.6 


7.3 


58.1 


Parent block quota sample 


258 


31.0 


5.4 


63.6 


Nonparent block quota sample 


242 


51.2 


4.2 


44.6 
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percent were other white, 4.1 percent black, and 44.6 percent Mexican-American. 
The higher proportion of other whites in this sample probably indicates that other 
whites living in the nonvoucher area are in general older and thus less likely to have 
children in school. It also suggests that the nonparent sample in the nonvoucher 
area is in fact more other white than Mexican-American, unlike the nonparent 
sample in the voucher area. 

Nonparents in the voucher area, the last of the block quota samples, were only 
29.5 percent other white, 5 percent black, and 65.8 percent Mexican-American. This 
high percentage of Mexican-Americans may well be explained by the classification 
of blocks on the basis of apparent Spanish surnames extracted from the Polk directo- 
ry and the fixed age quotas for males. A closer look at the demography of the voucher 
area in San Jose reveals that in this area of the city there are several Portuguese 
and Cuban families. These were included in the Spanish surname list rather than 
in the other white list in the block classification process. Thus, we sampled dispro- 
portionately more all Mexican-American blocks in this area than we might have if 
we had had a cleaner demographic classification of the blocks. This helps to explain 
the higher percentage of Mexican-Americans in the voucher nonparent sample. 

The comparison of the parent probability sample in the nonvoucher area and 
the parent block quota sample in the nonvoucher area reflects well on the block 
quota technique in terms of insuring demographic representativeness. We see that 
the percentages usually deviate by less than 3 percent. Some of this might be 
explained by the fact that the parent block quota sample is representative of the 
entire nonvoucher area, whereas the parent probability sample is representative of 
the entire catchment areas of six schools in the RECAP attendance project. Further 
analysis of these differences is necessary before any strong conclusions can be 
drawn. 



Appendix VI-C 
STUDENT TRANSFERS 
by Robert T. Riley 



This appendix: 

• Describes the transfer process at different times during the year. 

• Defines student transfers not related to parents' exercising their voucher option. 

• Separates a school's enrollment into various components and specifies the so- 
called legitimate transfer population. 

• Analyzes an administrative transfer pattern that occurred in the summer of 
1972. 



THE TRANSFER PROCESS 

The time of the year structures the transfer process. When a parent chooses to 
transfer a child, it is assumed that he has accurate and up-to-date information about 
other program offerings and their availability (they have not been closed out or do 
not have a waiting list). A parent must also file a request fo^' transfer in the form 
of an application in the spring or "transfer request" at other times during the school 
year. Moreover, a parent must contact school officials either at the central oflice 
during the summer, or at the school during the year to transfer a child. This contact 
with school officials usually involves contact with a parent counselor who will inform 
the parent about program offerings and their availability and often will help the 
parent to file the transfer form. 

In summary, to transfer a child a parent must have acquired information about 
other programs, have contacted school officials directly or indirectly, and have com- 
pleted an administrative procedure. On the basis of these steps involved in the 
transfer process, transfers can be divided into three types: application transfers, 
summer transfers, and school year transfers. 

Application Transfers 

In the spring of each year, usually by the end of May, a parent must file an 
enrollment form for each child in a voucher school, which enrolls a child in a 
program for the next school year. A parent must complete this application, whether 
his child remains in the same program or changes programs. Many program changes 
that occur at this time are system related, i.e., involve graduation, etc. 

On the enrollment form a parent gives three program choices for his child; a 
parent with a child in a program during the current school year is guaranteed a 
place for his child in the same program for the next year. However, if a parent wishes 
to enroll his child in another program for the next year, ho gives three program 
preferences. Usually parents receive their first choice. 

During the application period, information about program offerings for the next 
year is accurate and up to date. Parents have received information about the various 
programs from the school system during the month prior to the application deadline; 
they do not have to seek it out. To recei 'e more information about programs they 
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contact the stafT of the school in which they have children enrolled or in whose 
catchment area they reside; they do not have to contact the central office. Parent 
counselors, who are available at the school during this time of year, have informa- 
tion about all available programs, not just the ones in that school. 

Application transfers occur at a time when accurate information 's readily 
available and thus require the least effort on the part of parents ana the least 
administrative effort and record keeping on the part of school system. Moreover, 
completion of application transfers requires a minimal amount of contact with 
school staff or parent counselors. 

Summer Transfers 

A summer transfer occurs during the period between June and September after 
an application has been filed. Parents seeking a transfer during the summer must 
go to the school system's central office, not to a school, to complete the process. At 
the central office a parent must complete a "transfer request" form that negates his 
application's choice. Parent counselors are available there with accurate and up-to- 
date information about the available program offerings. Some programs may be 
closed out because thifey are filled, and others may have been eliminated. Seeking a 
transfer in the summer requires more effort than at the time of application, and the 
range of choices may be reduced. 

School Year Transfers 

Transferring a child during the school year requires the completion of a transfer 
request form and contact with a school official or parent counselor at the school, not 
at the central office. Information about the range of all program offering is less 
accurate and less available in the school year than at the time of application or 
during the summer. If a parent changes schools as well as programs, then he may 
have to contact the school staff at two schools. Transfers during the school year also 
involve disruption of the child's daily routine. Changing programs and, in particu- 
lar, changing schools means that a child's peer group will change, and the child may 
feel initially socially isolated. 

In summary, application and summer transfers require less effort and involve 
less disruption than school year transfers. The transfer process is more demanding 
in the school year. As a result it is reasonable to expect the transfer activity to be 
greater at the time of application and during the summer than during the school 
year. 



NONVOUCHER-RELATED STUDENT TRANSFERS 

Not all transfer activity can be attributed to parents exercising their voucher 
options to change the programs or schools of their children. A legitimate transfer 
involves a change in programs by a student that can only be attributed to parents 
taking advantage of their voucher option, not to other factors such as intra-district 
mobility, graduation, program discontinuation, or administrative decisions by 
school officials. 



82 



SYSTEM-RELATED TRANSFERS 

Most student transfers unrelated to parents' exercising their voucher option 
occur over the summer or because of residential mobility; a few result from adminis- 
trative decisions of school officials, and some appear in the data files because of 
clerical errors. Transfer activity over the summer usually involves the structure of 
the school or program — a student has completed its highest grade and must enter 
a new program in the fall. The time of year affects the definition of many transfers 
as voucher related or not; most student transfers that occur during the summer are 
system related, i.e., they depend on the structure of schools and existed before the 
advent of vouchers. The following system-related transfers are of interest: 

Graduation 

These are transfers that occur when a student completes an elementary school 
and enters a middle school. 

Feeder Schools 

Nonvoucher feeder schools provide the catchment areas for some of the voucher 
middle schools. Students enrolled in nonvoucher elementary schools last year who 
have had to transfer into middle schools participating in the demonstration this year 
are called "nonvoucher feeder school entrants." 

Program Discontinuation 

A school may discontinue a program and force all children to enter new pro- 
grams for the next year. This has not occurred in Alum Rock, but it is a possibility. 

Special Education 

In the voucher schools about 200 students are in special education classes. 
Transfers involving movement to or from a "special ed" class are not counted as 
"legitimate" transfers. 

Kindergarten Graduation 

Some voucher schools offer a program called kindergarten. After a child com- 
pletes his year in kindergarten, a parent has to choose a new program. However, if 
kindergarten is a grade offered as part of a larger program, and a parent chooses 
to enroll his child in another program's first grade, then the transfer would be a 
voucher-related transfer. 

Kindergarten 

Students who enter kindergarten are not considered "legitimate" voucher trans- 
fers. 



MOBILITY RELATED CHANGES 

Families change residences continually during the school year; some residential 
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mobility occurs within the confines of the school district; other families move into 
the school district from other communities, and some families move from the district 
to other communities. Student transfers associated with residential mobility affect 
the enrollments of the voucher schools, but these transfers cannot be attributed to 
vouchers. 

Intradistrict Mobility Changes 

When a family moves within the school district beyond the catchment area of 
the school associated Vv^ith its residence, then the parent has the option of leaving 
his child in the school associated with his old residence, or transferring his child to 
the catchment area associated with his new residence. In this case, transferring a 
child to the new school would not be a legitimate transfer, if it is only done for 
mobility reasons, even though the child's school may change. And, if a parent leaves 
his child in his old school, it might be considered a legitimate transfer. 

New Entrants 

These are families that move to Alum Rock and have to enter a child in a school. 
They are transfers from outside the district. 

Administrative Transfers 

The actions of school officials account for some types of student transfer activity. 
Some administrative transfers result from the actions of officials directed toward 
schools, while others result from the actions of school officials directed toward 
parents directly. 

Catchment Area Changes 

If school officials change the catchment area of a school, it is possible that some 
residents of nonvoucher catchment areas will find themselves in voucher catchment 
areas with the options of education vouchers now open to them. School transfers of 
this type can only occur in the summer. 

"Out Sneakers" 

Some parents in the voucher area have made arrangements for their children 
to attend nonvoucher schools even though they should be enrolled in voucher 
schools. 

"In Sneakers" 

In Alum Rock it appears that a limited number of families living in the non- 
voucher area have enrolled their children in voucher schools, through special ar- 
rangements with the school district, or through misrepresentation of their ad- 
dresses. 

Provisional Placement 

Provisional placement students may be temporarily placed in programs, i.e., 
they may be placed on a waiting list for some program if they did not receive the 
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first program choice, and provisionally placed in another, until space becomes avail- 
able in the first-choice program. When this happens, because of the recordkeeping 
system it first appears as if a transfer has been made by a parent, although the school 
has brought about the transfer. 

A schoors 1973 enrollment can be divided into eight components, taking into 
account the differences in transfer types previously described. These are: 

• Stayer: A student in the same school as the previous year without a program 
change. 

• Program changv^r: A student in the same school as the previous year with a 
change in program related to the use of vouchers. 

• School changer: A student who changed school because of parental utilization 
of a voucher option. 

• Kindergarten: A student entering kindergarten. 

• Graduation: A student in a new program because in the previous school year 
he was enrolled in a school or a program's highest grade. These may be kinder- 
garten graduates. 

• Feeder: A student scheduled to enter voucher middle school from nonvoucher 
elementary school due to middle school catchment area definitions. These stu- 
dents did not have to enter the middle school associated with their nonvoucher 
elementary school, but could select any program available for their grade level. 

• New entrant: A student from a fiamily who has moved to the district from 
another school district and is not enrolled in kindergarten. 

• In-sneaker: A student who resides in the catchment area of a nonvoucher 
school, not a middle school student, who attends a voucher school through a 
special arrangement. 

Separating a school's enrollment into these components serves several purposes: 

1. It demonstrates that several factors affect changes in a school's enrollment other 
than voucher-related transfers. 

2. It provides an estimate of the inmigration into the school district. 

3. It shows how system-related changes, graduation, etc., affect middle school en- 
rollments. 

4. It indicates the extent of within-school transfer activity by program changers. 



ADMINISTRATIVE TRANSFERS IN THE SUMMER OF 1972 

One particular type of administrative transfer that occurred in the summer of 
1972 clearly shows how the level of transfer activity associated with parents' exercis- 
ing their voucher options may be artificially inflated by the decisions of school 
personnel. 

In the summer of 1972 a distinctive transfer pattern emerged: 

1. Parents applied for and were assigned to the Cassell school. 

2. Over the summer parents had their children transferred to the McCollam 
school. 

3. By the time school began, students were back in the Cassell school. 

This transfer pattern characterized 80 students in 55 families. At first glance it 
seems that a significant number of parents were utilizing their voucher option to 
pressure the Cassell school system and that Cassell school responded to parental 
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pressure to draw back students it had lost to the McCollam school. This was not the 
case. The school system had reassigned the students, not once, but twice. When 
parents were asked about this change in the fall survey they claimed they neVer 
changed their children's school. This is a good example of the way in which estimates 
of the extent of transfer activity can be inflated. Whenever possible, estimates of 
transfer activity by parents and students have to be corrected for transfers not 
directly related to the voucher option. In particular, student transfers that resulted 
from administrative decisions of school officials have to be eliminated. 



Appendix VI-D 
SOURCES OF INFORMATION ABOUT VOUCHERS 
by R. Gary Bridge 



This appendix analyzes in some detail the sources of information about the 
voucher program that parents used. The purpose is to find out how informed parents 
became informed, as a function of ethnicity and social class. Only "aware parents" 
are included in this analysis: that set of parents who knew about the voucher 
program in Alum Rock. 

As mentioned earlier, 83 percent of the parents knew about the voucher model 
in Alum Rock in the fall of 1972. Eleven possible sources of information were 
identified: 

1. Radio and TV 

2. Newspapers 

3. Materials received from schools 

4. School bulletins 

5. PTA meetings 

6. Parent meetings 

7. Board of Education meetings 

8. Talks with children 

9. Talks with other parents, relatives, friends, and neighbors 

10. Talks with teachers and principals 

11. Talks with school counselors and advisors 

Parents were asked to indicate which sources they used. The average voucher parent 
received information from four sources, but the number of sources used, like aware- 
ness of the voucher model, was related to education and ethnicity. Anglo respond- 
ents used the most sources of information (4.6), and Mexican-Americans the fewest 
(3.2). 

Table VI-D.l indicates the frequency with which parents used each source of 
information. The one most frequently cited was printed school material; over four- 
fifths of the respondents (81 percent) learned something about vouchers from cflScial 
school publications. (The information distribution system in the first year relied on 
delivery of information in person by parent counselors to each household; it became 
known as the "doorstep" delivery system.) 

About 45 percent of the parents cited talks with teachers and neighbors. About 
a third learned about vouchers through talks with children, talks with parent 
counselors, and attendance at PTA and parent meetings. Radio and TV provided 
information to about one-fifth of the parents. Board of Education meetings were the 
least used source of information; only 6.7 percent of aware parents attended any of 
those meetings. 



Personalization and Involvement 

The first column of Table VI-D.l groups the eleven sources of information about 
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Table VI-D.l 



PARENTS' 


SOURCES OF INFORMATION ABOUT VOUCHERS, 
(In percent) 


BY ETHNICITY, 


1972 














Mexican- American, 
Interviewed in; 


Category 


Specific Source 


Total 
(495) 


Anglo 
(240) 


Black 
(56) 


English 
(143) 


Spanish 
(56) 




Radio and TV 
Newspapers 


19.6 
43.5 


23.2 
49.0 


16.0 
37.5 


15.4 
39.9 


17.9 
35.7 


School publlcaclons 


School materials 
School bulletins 


81.0 
49.6 


82.0 
57.7 


84.0 
39.3 


82.5 
44.1 


69.6 
39.3 


Formal meetings 


PTA meetings 

Parent meetings 

Board of Education meetings 


29.4 
31.5 
6.7 


34.0 
37.8 
8.7 


17.9 
25.0 
3.6 


28.0 
23.8 
1.4 


25.0 
30.4 
1.8 


Talks with 

nonschool people 


Talks with children 
Talks with friends 


25.8 
42.3 


28.2 
47.7 


21.4 
44.6 


23.8 
37.0 


25.0 
30.3 


Talks with 
school staff 


Talks with teachers, principals 
Talks with counselors 
— ., J 


46.8 
31.5 


48.5 
37.8 


48.2 
25.0 


41.9 
42.0 


50.0 
39.3 



NOTE: The question put to the parents read, "Now we would like to find out where and 
how much information you received about the voucher system and the different programs 
being offered by the schools." 



vouchers into five categories according to their degree of personalization. Of course, 
a correlate of personalization is ac^ii;^ involvement, and these two dimensions can- 
not be pulled apart easily. For example, it is easier to ignore a newspaper article 
than it is to ignore a speaker in a face-to-face conversation; and, in addition to 
demanding more attention, face-to-face communications can be aimed precisely at 
the listener or changed according to the listener's reactions. Newspapers, in con- 
trast, are aimed at a general audience, and direct interaction is impossible. At one 
end of the personalization-involvement dimension we have information gathered 
from the niass media: newspapers, radio, and TV. These are the least personalized 
and involving sources of information. At the other end are face-to-face conversa- 
tions, the most personalized and involving type of communication. We have broken 
this general category into two subcategories: talks with nonschool people (e.g., neigh- 
bors, children) and talks with school staff (e.g., teachers, principals, counselors). 

When we use these five categories instead of the original eleven basic sources, 
we find (Table VI-D.2) that printed materials from the schools were clearly the most 
widely used source of information about vouchers (86.7 percent), and formal meet- 
ings the least used. 

The ethnic groups varied in how manysources of information they used and, in 
general, they also varied in the types of information they received; however, they 
differed very little in their reported exposure to the two major sources of informa- 
tion, talks with teachers (and principals) and printed school materials. 

The first thing that can be seen in Table VI-D.l is that with the exception of one 
source (talks with teachers), Anglo parents were the most likely to have used a 
particular source. They were more likely than other parents to have learned about 
vouchers through radio and TV broadcasts, newspapers, school bulletins, attend- 
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Table VI-D,2 

PARENTS' INFORMATION ABOUT VOUCHERS, BY INFORMATION 
CATEGORY AND ETHNICITY, 1972 



Category 


Total 
(495) 


Anglo 
(240) 


Black 
(56) 


Mexican-American, 
Interviewed in; 


English 
(143) 


Spanish 
(56) 


Mass media 


47.6 


53.1 


41.1 


43.4 


41.1 


School publications 


86.7 


88.0 


87.5 


88.1 


76.8 


Formal meetings 


40.9 


47.3 


28.6 


35.7 


39.3 


Talks with nonschool people 


53.8 


57.3 


55.4 


51.0 


44.6 


Talks with school staff 


65.9 


68.0 


64.3 


61.5 


69.6 



ance at parent and PTA meetings, Board of Education meetings, and talks with their 
children and friends. For two categories only — talks with counselors, and school 
materials — Mexican-Americans and blacks, respectively, were about as likely to use 
that source as were Anglos. 

It seems logical that most Mexican-Americans who speak little or no English 
would be prevented from easy access to some sources of voucher information (e.g., 
radio and TV broadcasts are mostly in English), and if language is indeed a con- 
straint on communications, we should find that English-speaking and non-English- 
?r >' nng Chicanos used markedly different sources of information. The data in 
*a^• e VI-D.l do not support this notion, however. Parents interviewed in Spanish 
wrvi*e just as likely as those interviewed in English to have talked with school 
personnel, to have attended PTA and parent meetings, and to have learned about 
vouchers through radio and TV. In fact, those who speak predominantly (or only) 
Spanish in the home were more likely to have talked with teachers than were 
English-speaking respondents. Language may have been a constraint only in rela- 
tion to the printed word: printed school materials, school newsletters, and newspa- 
pers. 



MOST HELPFUL AND LEAST HELPFUL INFORMATION 
SOURCES 

The frequency of a person's exposture to an information source does not neces- 
sarily reflect its influence on his decisionmaking. To learn which sources they valued 
most, parents were asked to review the original list of eleven sources and answer 
which they found most helpful (Q40B) and least helpful (Q40C). (This item was asked 
only of respondents who reported having used more than one source of information. 
Of course, for those parents who only cited two sources, the term "least helpful" 
source is probably not very meaningful.) Table VI-D.3 summarizes their responses. 
Printed school material ranked highest in helpfulness, with 30.4 percent of the total 
vote. Talks with teachers, principals, and counselors also ranked high (around 20 
percent). The other sources received scattered votes. 

Newspapers, talks with friends and children, and — surprisingly — printed school 
materials were all rated as least helpful by at least 14 percent of the parents. Why 
printed school materials should be rated so high on both the most and least useful 
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Table vVd.3 

PAK£NTS* RATINGS OF MOST AND LEAST HELPFUL SOURCES 
OF INFORMATION, BY ETHNICITY, .1972 





Most 


HpI nf til 


S ourc£ 




Least 


Helpful Source (Q40C) 










Mexican- 








Mexican- 




Total 


White 


Black 


American 


Total 


White 


Black 


American 


Source 


(447) 


(220) 


(52; 


(175) 


(419) 


(211) 


(47) 


(161) 


Radio and TV 


1.1 


0.9 


3.8 


0.6 


10.5 


10.9 


6.5 


11.2 


Newspapers 


2.9 


2.7 


3.8 


2.9 


18.4 


18.5 


10.1 


18.0 


School materials 


30.4 


30.0 


34.6 


29.7 


14.3 


11.4 


17.0 


17.4 


School bulletins 


4.9 


9.1 


5.8 


5.7 


9.5 


9.5 


12.8 


8.7 


PTA meetings 


4.3 


5.9 


0.0 


3.4 


5.0 


4.3 


0.0 


7.5 


Parent meetings 


6.5 


7.3 


3.8 


6.3 


2.9 


3.8 


2.1 


1.9 


Board of Education meetings 


1.8 


1.8 


0.0 


2.3 


0.7 


0.9 


0.0 


0.6 


Talks with children 


1.8 


1.4 


1.9 


2.3 


14.8 


18.5 


14.9 


9.9 


Talks with friends 


6.0 


7.3 


7.7 


3.7 


16.2 


14.2 


23.4 


l'')«8 


Talks with teachers, principals 


19.7 


19.1 


17.3 


21.1 


4.1 


4.3 


2.1 


/^.3 


Talks with counselors 


19.9 


19.5 


21.2 


20.0 


3.6 


3.8 


2.1 


3.7 



lists is not clear; 30 percent of the sample thought this was the best source of 
information, and here we see that half that number, 15 percent, felt just the oppo- 
site. The explanation may be partly cultural. Mexican-Americans interviewed in 
Spanish were almost twice as likely as whites to rate printed materials as least 
helpful; and fewer of them than of other ethnic subgroups rated school materials as 
most helpful. 

It is not safe to attribute those differences to language, however — unfamiliarity 
with English — since the schools furnished abundant material printed in Spanish. 
The answer may therefore be not language, but literacy. 



ADEQUACY OF INFORMATION ABOUT VOUCHER SCHOOLS 

Did aware parents feel they had enough information to 
make informed program choices? 

Aware parents were first asked: 

Q40D. When it came time to make a choice, did you find you had 
enough information about the different programs being 
offered in the school you selected for your child(ren) or would 
you have liked to have more information about the program 
offered there? 

About half the respondents said they had enough information (52 percent); 26 per- 
cent said that they would have liked more about the school they selected; and 23 
percent simply could not say. 

Parents responded to a similar question about schools not selected for their 
children: 
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Q40E. What about the other schools in the voucher system — did you 
fSeel you had enough information about the different programs 
they were offering or would you have liked to have more 
information about the programs offered at the other schools? 

Parents rated the adequacy of their information about other schools in about the 
same way they rated their information about the school they selected for their 
children; the correlation between these two ratings was 0.86. Of the respondents, 43 
percent were satisfied with the information they had about nonselected schools; 26 
percent would have liked more information; and about a third had no feelings one 
way or the other. 

There were no significant differences in ethnicity and education in aware par- 
ents' responses to the foregoing questions. 



INFORMATION DISii.iBUTION 

The Alum Rock school system was mainly responsible for distributing informa- 
tion ab'^ut vouchers and explaining the voucher system to parents. That function is 
very important. On one hand, the actual effectiveness of this operation directly 
affects the operation of the voucher system by providing or failing to provide infor- 
mation vital to parents if they are to make rational program choices; and on the 
other hand, the perceived effectiveness of this operation by parents contributes to 
the overall community evaluation of vouchers. 

Do parents think the school system has done a good job 
explaining the voucher system and its operation, and 
how accurate is their information? 

The answer summarizes parents' perceptions of the school system's job perform- 
ance in providing information. Parents were asked: 

How good a job do you feel the school system has done in 
explaining the voucher system and how it works — do you 
think they have done a very good job, a good job, a fair job, 
or a poor job? 

Of the respondents, 65.4 percent said the system had done a "good" or 'Very good" 
job. Whites were more satisfied than Mexican-Americans or blacks. Educational 
differences had no significant effect. 

To probe in more detail parents' perceptions of the way Alum Rock provided 
information about vouchers, items were selectively asked of parents about personal 
agents of information distribution. Parents who ei iployed teachers and principals 
{Q40E), parent counselors (Q40F), and friends (Q40G) as sources of information were 
asked to evaluate "how good a job they did in explaining the voucher system." Only 
92 parents felt qualified to evaluate the effectiveness of teachers and principals in 
this respect, and only 90 felt qualified to evaluate parent counselors — less than 20 
percent of the aware parents in each case. These selected parents thought very well 
of the two sources of information: over 80 percent believed teachers and principals 
did a good job, and over 70 percent believed that parent couselors did a good job. (So 
few parents (34) responded to the item on friends as a source that it was meaningless 
to analyze it.) 
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In short, parents who felt qualified to pass judgment gave a positive evaluation 
to the information they received about vouchers and to its distribution. It is impor- 
tant to note, however, that most parents did not believe they were knowledgeable 
enough to make such judgments. One would have expected parents' knowledge 
about vouchers and their operations to increase during the school year. 



ACCURACY OF INFORMATION ABOUT VOUCHERS 

Aware parents generally believed they had enough information to make good 
choices^among school programs, but: this does not mean they actually did. To get an 
idea of the accuracy of their information, we disked parents who pays for school 
transportation for children in voucher programs (Q48), and we asked them if they 
agreed or disagreed with the following statement about the schools' transfer policy: 

Q55B. Under the voucher system a parent can change the school his 
child attends at regular times during the school year. 

About three parents in five knew that the school system provides free transpor- 
tation as necessary for children attending voucher schools distant from their neigh- 
borhoods (58.7 percent) and only one parent in two knew that schools or programs 
can be changed at regular times during the year. There was a high level of ''don't 
know" responses on the latter item (17.9 percent) among aware parents; this inflated 
the percentage of parents who had misinformation as opposed to no information. 
Thus, roughly one parent in two had correct information and one in six no informa- 
tion. At the time of the 1972 parent survey (November), however, transfer rules were 
still in flux, so it is difficult to interpret the responses to Q55B. 



Appendix VII 

DEMOGRAPHIC CHARACTERISTICS OF ALUM ROCK 

TEACHERS 



The following figures show demographic characteristics of Alum Rock's teachers 
by school. 



Under 30 



Percent 
30-49 



Over 50 



District 
Old Voucher 
New Voucher 
Nonvoocher 

Old Voucher 
Cossell 
Goss 

McCollam 

Meyer 

Miller 

Polo 
New Voucher 

Arbuckle 

Conniff 

Dorsa 

F ischer 

Hubbard 

May fair 

Sheppord 
Nonvoucher 

Cureton 

George 

Linda Visto 

Lynda I e 

A/othson 

Painter 

Rogers 

Ryan 

San Antonio 

Shields 

Slanoker 



25 
24 
22 
28 

15 
36 
33 
21 
27 
14 

24 
W 
22 
25 
17 
34 
13 

0 

29 
12 
18 
54 
0 
50 
22 

31 
14 
39 



70 



52 



58 



72 



79 



64 











61 






64 . 




64 






56 



92 



39 



XI 



68 









71 


























55 














59 






70 





57 



14 

12 
14 
16 

15 
12 
9 
14 



5 
(9 
13 
20 

8 

7 
17 



24 
18 
5 
8 

20 
39 

13 
18 



Number of 
Respondents 

608 
154 
204 
250 

26 
25 
24 
29 
21 
29 

21 
16 
23 
44 

24 
29 
47 

14 
35 
25 
22 
37 
13 
20 
23 

16 

22 
23 



SOURCE: Leinwond Associotes {doto 
monogement controctor) , 



Fig. VIM— Employment of teachers within Alum Rock, by age 

(fall 1972) 
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Percent 



Ma le 



F ema I e 



DIstf ict 
Old Vauc'ier 
New Vouclier 
Nanvouclier 
Old V ouclier 
Cassell 

Goss 

McCol lorn 

Meyer 

MMIer 

Polo 
New Voucher 

Arbuckle 

ConnI ff 

Dor so 

F Ischer 

Hubbard 

Mo y f o i r 

Sheppard 
Nonvoucher 

Cureton 

George 

Lindo V^sta 

Lynda I e 

Mathson 

Palmer 
Rogers 
Ryan 

San Antonic 

Shields 

Slanaker 




10 
12 
17 
30 
12 
21 
43 



90 
88 
83 
70 
88 
79 
57 




Number of 
Respondents 

612 

155 

204 
253 

27 
25 
24 
29 
21 
29 

21 
16 
23 

44 
24 

29 

47 

15 
35 
25 
22 
38 
13 
21 
23 
16 
22 
23 



SOURCE: Leinwand Associates (data 
management cantractor) . 



Fig. VII-2 — Employment of teachers within Alum Rock, by sex 

(fall 1972) 
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4 


George 


6 
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0 
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11 
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0 
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4 
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0 
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SOURCE: Alum Rock Union School District, "October 1972 
Racial and Ethnic Doto (Revised 12/13/72)." 
NOTE: Figures include teochers plus other certificoted stoff 
at schools. 
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Fig. Vn-3 — Employment of Spanish-surnamed and black 
teachers within Alum Rock 
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Respondents 
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21 
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SOURCE: Leinwand Associates (data 
management contractor). 



Fig. VII4— Teachers' salaries in Alum Rock, by school 
(fall 1972) 



Appendix VIII-A 



METHODOLOGICAL ISSUES IN MEASURING STUDENT 

ACHIEVEMENT 

by Pierce Barker 



INTRODUCTION 

This section describes some of the problems associated with testing in a large- 
scale social demonstration. It looks at the administrative and clerical problems of 
conducting the testing as well as at the problems of interpretation and comparabili- 
ty caused by out-of-grade-level testing, lack of control groups, and the high mobility 
of the student population. Because of the attendant problems, it is premature to 
draw any but the most tentative conclusions about achievement in the first year of 
the voucher demonstration. (Preliminary analyses of available data yield somewhat 
conjflicting evidence about the performance of voucher students in the first year of 
the demonstration compared with other students and other years. However, the 
effects are small in magnitude, regardless of direction; i.e., they do not seem to have 
much practical significance. More definite conclusions must await further data and 
analyses.) Further analysis of the achievement data depends upon resolving the 
problems discussed in this section. 

Measurement of student educational outcomes in the Alum Rock Voucher Dem- 
onstration comprises two components: achievement testing and affective measure- 
ment. As usually happen? in educational evaluation, however, the emphasis is on 
achievement testing, although it must be remembered that in actuality achievement 
testing measures only one sector of a complex space in which students may be 
expected to change. Achievement tests do not measure certain dimensions of cogni- 
tive functioning (e.g., creativity) nor do they measure the complex area generally 
designated as affective. Measurement of both domains is difficult, although a few 
primitive instruments for measuring some dimensions of affect are available for 
elementary school children. 

Rand arranged for the administration of the Metropolitan Achievement Test 
(MAT) to all students in the voucher demonstration schools in the fall of 1972 and 
spring of 1973. In addition, we administered various affective test batteries to sam- 
ples of voucher students. Rand did no achievement or affective testing in nonvoucher 
schools. Our main purpose in evaluating the voucher demonstration through 
achievement testing is to chart the trend of test scores over time, both within Alum 
Rock and between Alum Rock and other reasonable comparison groups. On the basis 
of data gathered directly from the voucher evaluation, we are limited to charting 
trend lines from Year 1 as baseline to Year n within the district. Other comparisons 
depend upon data gathered and stored independently of the Rand study. For exam- 
ple> using test data from periods prior to the demonstration, we may be able to 
approximate a multiple time-series analysis. With test data on nonvoucher schools 
within the district, parallel trends can be examined; this would allow us to control 
to a large extent many between-district factors. Still other data may permit a 
comparison of voucher schools with other, similar districts. 

Analyses or displays of this kind would at best seek to determine whether or not 
the voucher demonstration as a whole had a discernible effect on achievement 
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scores; these efiects would, of course, be at the district/demonstration level. There 
is also interest in internal analyses of, for example, the differential effects of pro- 
grams or types of programs; given the exercise of suitable caution in drawing infer- 
ences, these comparisons may also be possible. In the same vein, given suitable 
sample size, we might in the long run examine more particular effects; for example, 
how did different programs or program types affect different types of students (e.g., 
students of different ethnic backgrounds or compensatory voucher students). 

For the most part, the analysis of the first year's achievement data is aimed at 
resolving a number of testing issues that are seemingly endemic to demonstrations 
of this sort. What follows is a discussion of many of these issues, including tentative 
explorations of the adequacy of the data base. A definitive analysis of the effects of 
the demonstration as a whole, as well as more particular effects, depends upon the 
resolution of problems related to test administration and other issues to be dis- 
cussed. Details of these and other analyses appear in Rand report R-1497-NIE, 
forthcoming. 



PROBLEMS OF ACHIEVEMENT TEST ADMINISTRATION 

Testing at Achievement Level 

On the whole, student^; in Alum Rock do not achieve at grade level; for example, 
many students enrolled in the 5th grade might well be able to read on a level 
expected of the modal or "average** student in the 3rd grade. Looking at the matter 
the other way around, it might easily happen that 4th grade students were reading 
over a range of, say, 1st to 6th grade, with comparatively few at or above the 4th 
grade level. 

In testing situations, then, an important question is, "What level of test shall 
students be asked to take?** If a 4th grade student can read only at the 1st or 2nd 
grade level, most of the student*s answers to a 4th grade reading test are likely to 
be guesses; and if this is true, the score probably has no meaning. But perhaps the 
student is tested on a level appropriate to the student*s achievement level, whatever 
it may be, without regard to nominal grade level. This means that each grade may 
have a wide range of achievement levels among its students, and therefore a wide 
range of tests should be offered within a grade. 

Evaluation of educational programs, however, is generally concerned with the 
impact of programs on groups of students, so that evaluators are mainly interested 
in summary statistics, e.g., means, medians, and measures of variability. These 
statistics, of course, depend for meaning in some sense on the assumption that they 
summarize numbers that are the "same thing.** More concretely, it is not obvious 
what an average score means if it is based on scores from, say, four or five different 
tests, even though they may all be testing "reading." 

The MAT was selected for use in the voucher demonstrati6n for a number of 
reasons. The University of California at Los Angeles Center for the Study of Evalua- 
tion has published a comparison of the usefulness of various available achievement 
tests in the elementary grades.^ In terms of measurement validity, examinee appro- 
priateness, administrative usability, and excellence of technical norms, the MAT 
was rated consistently higher than all other tests, and was also recommended by 

• R. Hoepfner (edJ, CSE Elementary School Test Evaluation, Center for Study of Evaluation, UCLA 
Graduate School of Education, Los Angeles, 1970. 
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several authorities in the educational field. In addition, the MAT is mandatory in 
the Title I program, and comparisons are thus possible between voucher and Title 
I schools. Finally, and not least important, the Alum Rock school personnel general- 
ly accepted the MAT. 

Teachers in Alum Rock were insistent that giving each student a grade-appro- 
priate test would be useless if not harmful in a great many cases. Many teachers had 
been working hard to give students material tailored to their own achievement level, 
in an effort to provide them with success experiences. Since the achievement tests 
were not administered until early November, teachers were especially reluctant to 
risk undoing this work by exposing students to a failure situation. The result was 
that a great deal of out-of-grade testing was done, and it remains to be determined 
whether the scores can in fact be compared meaningfully. 

The publishers of the MAT anticipated the problem of out-of-grade testing by 
providing a transformation of raw scores, which theoretically enables a user to place - 
the score from any collection of subtests in the same area on a common linear scale. 
However, this was a theoretical exercise; and it also is not clear how appropriate the 
scoring system is for a group of largely educationally disadvantaged students who 
are taking an unusually wide range of tests. 

One partial solution is to attempt to verify the validity of the published transfor- 
mations for Alum Rock students. Accordingly, a small study was done on all 3rd 
grade students in Alum Rock to determine for two adjacent test levels whether or 
not the transformations seemed to work. The results are complex, but it seems 
tentatively reasonable to say that the transformations seem valid, with a not enor- 
mous margin of error. But additional work is needed to determine whether this 
result will hold over wider ranges of test levels. 

Administrative Problems 

Some administrative difficulties developed during each testing session. For ex- 
ample, in the fall of 1972 teachers had to record the students' names, ID numbers, 
and ethnicity on the answer sheets and booklets before the test was administered. 
As this procedure appeared to absorb a great deal of the teachers' time and many 
teachers were already commenting on the many extra burdens imposed on them by 
the voucher demonstration, Rand decided to procure clerical assistance to perform 
these tasks in the spring. Thus, for the spring administration, these data were 
entered on each student's answer sheet and test booklet before these materials were 
delivered to the schools. Another problem that developed during the spring testing 
involved the actual test booklets — a number of the MAT booklets had missing or 
roside-down pages. The manufacturer was informed of this problem, but the teach- 
ers were nevertheless inconvenienced and the testing process was disrupted. 

Three test coordinators were hired for spring testing to provide any assistance 
the teachers might need. Each coordinator was assigned to work closely with two 
voucher schools. The coordinators initially contacted the school principals and sche- 
duled meetings with each mini-school faculty or a faculty representative to discuss 
test procedures, scheduling, or any problems that the teachers felt might arise. 
Before testing sessions began, the coordinators delivered the test materials to the 
schools; when testing was completed, they picked the materials up and returned 
them to the Rand office to be sorted, packaged, and mailed to the scoring service. 
Although the coordinators spent a great deal of time in the schools, they did not 
formally observe the testing conditions or procedures used. Thus, although their 
presence may have lessened the burden on the teachers, it is likely that they had 
no impact on the actual testing conditions or procedures that the teachers followed. 
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One dilliculty that teachers faced in administering the achievement tests in- 
volved the multilevel testing within a classroom. Although there were no ofHcial 
Rand observers to monitor the testing during either the fall or spring administra- 
tions, teachers have mentioned during conversations with Rand staff members that 
they attempted to solve this problem by switching portions of their classes with 
other teachers who were testing at the same grade level, so that each teacher would 
only be required to administer one level at a time. 

Test scoring was accomplished in a series of steps during both the fall and spring. 
When the completed test booklets and answer sheets had been returned to the Rand 
San Jose ollice. staff members sorted the test sheets, then scored those portions of 
the tests that required handscoring. The levels involved were the spelling and/or 
math subtests of the Primer. Primary I or II, and Elementary test levels. The tests 
were then packaged and sent to National Computer Systems (NCS) in Minneapolis, 
Minnesota, for machine scoring and processing. 

Some problems associated with the scoring process for the fall test administra- 
tion should be noted. Although the tests were delivered to NCS early in December, 
the results were not returned until mid-February because NCS received a great deal 
of business at this time and was not prepared to handle such a large volume at once; 
in addition, although the MAT test handbooks state that any test level may be given 
to any student, the conversion tables (raw score to standard score and grade equiva- 
lent* that NCS used were not adequate to handle the type of out-of-level testing done 
in Alum Rock. Some additional programming was required to process the tests. 
Although NCS did send the results to Rand in mid-February, NCS later notified 
Rand that due to a programming error, all total reading scores for Primary I and 
Primary II levels for the fall tests were reported from three to ten months lower than 
actual scores. NCS created a new tape with the correct data and sent it to Rand 
during the summer. 

The scoring problems proved irritating to the teachers, as many had hoped to 
be able to use the test results to identify areas where students needed help. Since 
the results arrived so late in the school year, they were probably unable to fulfill this 
expectation. Rand staff members held a meeting with representatives of each mini- 
school faculty in mid-P'ebruary to explain the test results and the reasons for the 
delay. 

In general, the fall testing encountered more problems than the spring test 
sessions. Many of the changes implemented in the spring were oriented toward 
smoothing out previous areas of difficulty. NCS appeared to improve its service 
considerably for the spring round of testing (apparently some of its earlier problems 
had also been eliminated). Work was carried on over the summer at Rand to try to 
further reduce testing problems — e.g., the development of the achievement place- 
ment test described below. In addition to improving the quality of the data, it would 
give teachers more opportunity to plan cooperatively when administering several 
levels of an achievement test. 



DEVELOPING AN ACHIEVEMENT PLACEMENT TEST 

In response to problems encountered in 1972-73, Rand developed a short place- 
ment test to aid teachers in selecting the appropriate test level for each student for 
the fall 1973 test administration. 

One may think of the process of assignment of students to test levels by teachers* 
informal judgments as the use of a kind of relatively unstandardized, implicit place- 
ment test. That is, one uses as an assignment criterion pnor data on the students 
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which is processed in unstated and probably idiosyncratic ways to produce an implic- 
it function predicting student performance.^ 

ft is easy to imagine a number of diHiculties with this process. Perhaps the most 
formidable, from our perspective, is this implicit pretesting process is not 
explicitly linked to the achievement battery we are using. Now, any achievement 
test is necessarily based upon a sample ihov/ever derived) of items thought by the 
test writer to represent, optimally or reasonably, achievement in a particular area; 
and this sample will not be the sample chosen by another tester: the correlation 
between two grade (k) tests of, say, reading, is not 1.00. For any .s/;(?ci/?Vachievement 
test, however, samples of items on parallel forms are more likely to be drawn from 
a more closely defined item universe than are items said to measure achievement 
in the same area that are sampled (constructed) by another tester. Presumably, 
then, performance on one form of a given test is likely to be a better predictor of 
performance on another form of the same test than is anything else at hand. Now, 
what we meanhy "achievement" in the present context is just a student^s score on 
a particular testing form. It seems to follow that the more explicit we can make the 
placement process and the closer we can link it to the achievement battery in use, 
the better our placement procedure is likely to be, where "better'' means assigning 
students to proper test levels. 

With this preface, we can begin to explore what we may call the theory of the 
placement test. The trouble with the usual achievement test is that each test level 
measures only within a fairly narrow range; and one gets only one guess at what 
the proper range for a given student should be. It is as if one had a set of scales, each 
of which would record accurately only for a limited range of weights; and, in order 
to know which scale to pick to get an acci/ra/e weighing for an individual, one had 
to know 6^/bre weighing approximately how heavy the individual was. To continue 
the analogy, one requires another scale, which weighs perhaps more crudely but 
over a wider range; and the crudeness of the preliminary scale is not really much 
of a drawback, since having u;ied it, one should have a much improved chance of 
placing the individual on the proper, more accurate scale among the set of accurate 
scales. The present problem is to find or create an achievement test that will func- 
tion much like the crude scale of the analogy: it will cover a wider range of ability; 
it will presumably not give as accurate a reading of the student's achievement as 
will the proper more sensitive test, but (if properly constructed) it should give an 
accurate indication of which more sensitive test to give. So how do we achieve this 
cruder but broader range test? As a first approximation, the best available answer 
seemed to be the following: 

1. Choose one of the parallel forms of the MAT that would not be used in actual 
testing. 

2. Pick from among the available subtests one which 

(a) Was as robustly correlated with other subtests across all test levels as possi- 
ble. 

(b) Had a relatively constant format across all test levels. 

(c) Was prima facie unifactorial in structure, such that items could be sampled 
with relative ease without concern for differing content or required skills. 

3. Choose from the items on this subtest a sample from each necessary test level 
such that the sample would be representative of the subtest at that level. 

4. Form from these samples a new; subtest by, as it were, laying the samples from 
test levels of increasing difliculty end to end, so that the new subtest would in 

^ P. E. Meehl, Clinical vs. Statuitical Prediction, University of Minnesota Press, Minneapolis, 1954. 
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lact cover a wider ability range than any of the proper subtests taken individual- 
ly, but would do so more crudely than any individual proper subtest if the proper 
subtest were at the proper level for the student. 

Upon considering the various subtests, we concluded that Word Knowledge (WK 
- vocabulary) fulfilled the above requirements better than the others. That is, the 
correlation of WK with other subtests has a median higher than that of any other 
subtest across all levels; its format is as nearly unilbrm across levels as possible; and 
it is prima facie a one-factor test. In short, from subtests available, WK looked more 
likely to work than any other. The question remained, would anything work? 
Theoretically, the answer is, "Sure it will!" In fact, there is at least one relevant (if 
somewhat oblique) precedent for this procedure in the individual (as opposed to 
^TOup) testing area.^ Before administering the Stanford-Binet, the tester gives the 
subject a wide-range oral vocabulary test to determine a rough floor and ceiling for 
the subject, thereby establishing a useful testing range. That, of course, is just what 
we are trying to do for group testing. 

However, the present effort is very much a theoretical exercise; that is, we must 
make a number of assumptions that may lack empirical support in the population 
at hand. Put another way, the field trial and the application of the instrument are 
identical: the pretest of the instrument, which in the normal course would provide 
data only to check the instrument itself, was also its first real-world application. 

Analysis of fall 1973 testing is in process, and it is too early to say whether the 
placement test in fact improved the proportion of useful scores, i.e., those that give 
the most accurate reading of a student's achievement. 



PROBLEMS OF COMPARABILITY AND INTERPRETATION 

Problems of Group and Data Comparability 

In the absence of formal comparison groups, comparisons for the voucher dem- 
onstration depend upon the definition of comparable groups, i.e., groups as similar 
as possible to the group composing the voucher schools. One such group might 
reasonably be nonvoucher Alum Rock schools. These schools will, of course, be 
similar in many ways to voucher schools, more similar perhaps than most others, 
but they do differ in that they did not elect to become voucher schools. This may or 
may not have a bearing upon academic achievement measures, but the question 
cannot be decided by fiat. A partial solution to this difficulty lies in what has been 
called the multiple time-series mode of analysis, which essentially charts trends for 
nonrandomized groups over periods of time sufficient to extend reasonably well on 
both sides of some arbitrary point of intervention.'* 

Even if these groups were sufficiently comparable, the problem remains of 
finding comparable data. For example, most or all of the schools in Alum Rock do 
achievement testing; some of this is done by Rand for evaluating the voucher demon- 
stration, some by the district in response to requirements of various state aid pro- 
grams, some in response to statewide school evaluation programs. However, these 
testing programs do not generally use the same tests. The problem, then, is to 
translate raw scores on one test into those of another. For example, we might want 

^ Robert Rosenthal, personal communicatir.i. 
D. T. Campbell and J. C. Stanely, Experimental and Qiiard- Experimental Designs for Research, 
Rand-McNally. Chicago, 1963. 
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to compare voucher with nonvoucher 6th gi'^^dt's over time, even though the voucher 
schools wcM'e tested part of the time with the MAT and previously with the California 
Test of Basic Skills (CTBS). and the nonvoucher schools are currently tested with 
CTBS and formerly in some cases with the Stanlbrd Achievement test (SAT). Compa- 
rability studies have been done, sometimes by the test publishers, sometimes by 
outside researchers such as Educational Testing Service (ETS), to attempt to equate 
scores on one subtest with those on a different but comparable subtest. 

As it happens, there are commonalities over groups in available data. For exam- 
ple, ^//schools in Alum Rock test their first, second, third, and sixth grade students 
with tests mandated by the State of California, so that comparisons between, say, 
voucher and nonvoucher Alum Rock schools for these grades may be done over time. 
Also, in 1972-73 Title I schools in Alum Rock, which are not in the voucher demon- 
stration, tested their students with the MAT, and starting in 1973-74 MAT was 
extended to all nonvoucher schools; so that these schools may be compared on MAT 
with voucher schools. The problem of divergent results appearing dependent upon 
which test scores are used is quite thorny, as appears in preliminary analyses 
elsewhere in this report. Still, these problems should not be insuperable, and exten- 
sive analyses of data sets of this type are contemplated; some results of such analyses 
will appear in Rand report R-1497-NIE, forthcoming. 



RELIABILITY OF TESTS IN ALUM ROCK 

Although test publishers commonly invectigate the reliability of their achieve- 
ment tests before releasing them for public use, it is a matter of prudence, and 
recommended by many publishers, to determine reliability under actual conditions 
of use. After defining a population of Alum Rock students as comparable as possible 
to the population used by the test publisher, we investigated the reliability of all 
subtests at all levels for both the spring and fall testing periods. For various reasons, 
we did not expect reliabilities to be as high for any local, fairly homogeneous popula- 
tion as they are for a national sample of sti^dents. Allowing for this, however, most 
subtests on the basis of use in Alum Rock seem to be sufficiently reliable for evalua- 
tion purposes. 



SOME PROBLEMS WITH COMPARING STUDENTS OVER 
.TIME 

Any generalizations we might hope to make about the voucher demonstration 
would ideally apply to "the students in Alum Rock schools,'' where this is understood 
to refer to the whole population of students now in school and other students like 
them. 

Suppose we wanted to study test score trends over a period of, say, three years, 
by following the same students and testing them at regular intervals. We might do 
this, for example, by choosing a random or stratified random sample of students and 
actually testing them, say, twice yearly for three years. (This would depend upon our 
ability to follow these same students over time, which would imply among other 
things that the students stayed in Alum Rock and stayed in the voucher demonstra- 
tion.) 

From the test data we have for this year, our best estimate is that the mobility 
of Alum Rock students is about 30 percent annually, which amounts to about 51 

O 
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percent over two years. This is consistent with available estimates from Alum Rock/' 
If we had a large enough sample to start with, the percentage leaving would not 
matter because there would be enough students sufficiently well distributed over 
classes of interest, as long as the students who left did not differ significantly on 
relevant characteristics from those who remained. 

Among relevant characteristics are such things as race and class indicators; but, 
other things equal, one expects students of lower socioeconomic status (SES) to be 
more mobile, i.e., that the more mobile students will differ in relevant ways from 
those who remain over a period of years. 

To determine whether this seemed likely to be t,rue in Alum Rock, we investigat- 
ed the deferences between students who were present in the Alum Rock voucher 
schools long enough to be tested in the fall and spring, those who left after the fall 
testing, and those who arrived after the fall testing. 

The results are complex and equivocal, and not easily summarized; full details 
appear in the report previously mentioned. Briefly, however, it can be said that 
differences appear in pre- and post-test scores that may or may not be sufficiently 
important to cast doubt upon comparisons and generalizations, Differences in pro- 
portions of ethnic groups seem small, while differences in economic or class indica- 
tors are larger and quite possibly significant. 



PROBLEMS IN ASSESSING INTERNAL EFFECTS 

In looking for ''effects" of the voucher demonstration on academic achievement, 
we might take at least two approaches. First, we could compare students in voucher 
schools with otlier students. But this is a very gross procedure; aflei all, the voucher 
demonstration as it has evolved is not, so to speak, a single treatment, nor are all 
the students alike. A number of variations within the demonstration might be 
expected to have different effects, or even different effects for different kinds of 
students. 

The most obvious problem associated with this way of looking at the matter 
arises when we try to look at gross classifications of students in isolation; for exam- 
ple, we might want to look at white students versus others. This may, in fact, be a 
fairly reasonable thing to do; for white and other students are both fairly well 
distributed over the various classifications into which we might sort students — say, 
grades, mini-schools, SES, and so on. 

If we took our second approach and wished to look at the effects of types of 
mini-schoo's in isolation, the problem would immediately become much more com- 
plex. Basically, this results from what is technically called ''confounding." Suppose 
one wished to test the efficacy of male as against female teachers in using a given 
teaching method, and one chose one male and one female teacher to use the method 
and assessed the results. The problem is already there, but to make it more obvious, 
suppose the male teacher were white and the female black. Suppose also that the 
results for the first teacher were clearly worse than those for the second, and we 
wanted to conclude that female teachers were better. Someone could reasonably ask 
how we knew that the difference was not in fact the result of a racial rather than 

Report to the Trustees of Alum Rock Union Elementary' School District. 1972. This report on achieve- 
ment testing noted a 30-percent loss between 1st and 2nd f(rades of students tested as 1st ^(raders. and 
a 50-percent loss between 1st and 3rd grades ofstudents tested as 1st graders, implying an annual loss 
rate of 30 percent from the initial cohort. 
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a sex difference, i.e., that we had shown that black teachers were better than white 
with this method. 

Of course, the answer is that we would not know, nor could we on the basis of 
a study so conducted, because sex and race would be completely confounded. Some- 
thing like this is to be found in the voucher demonstration; certain types of programs 
probably attract certain kinds of students, and it would not be simple to say whether 
differences were due to programs or students. Programs are not evenly distributed 
across schools, so that it would also be difficult to separate program effects from 
school effects. 

This is not to say that the problem is hopeless; only that it is difficult and 
demands very careful handling to avoid erroneous inferences. 



SUMMARY 

As the problems described above indicate, we are still far short of the ideal we 
may implicitly have in mind when we consider assessment of student achievement. 

Whatever the school system involved, it is never a wise idea to try to assess the 
effects of schooling simply by subtracting spring scores from fall scores and looking 
at the difference. However, given this, in what we might think of as the "average" 
school system, we might expect problems connected with the assessment to be 
manageable. 

In Alum Rock, it quickly became obvious that the problem of assessing achieve- 
ment is qualitatively different from that encountered in many school districts. The 
variety of educational offerings is probably wider in Alum Rock voucher schools 
than in most school districts in the country. The span of achievement within single 
grades and indeed classrooms is unusually wide, and this leads to severe problems 
in interpreting test scores. The mobility rate among a subset of students is high, 
which makes for problems in assessing the progress of students over time and 
generalizing the results of any analyses to the voucher schools as a whole. 

This is not to say that meaningful assessment of student achievement in Alum 
Rock is impossible; but efforts at assessment, prior to careful examination of the data 
base and painstaking efforts to resolve the problems with that data base, would be 
premature and in many cases quite misleading. 

Finally, it does not seem prudent to try to draw more than very tentative 
conclusions from first-year data. This would be true in any case, simply because more 
than two time points are in general required for assessing trends; but the presence 
of unusual problems must reinforce the need for caution in assessing achievement. 



Appendix VIII-B 



PRELIMINARY ANALYSIS OF ACHIEVEMENT TEST 
SCORES IN ALUM ROCK VOUCHER 
AND NONVOUCHER SCHOOLS, 1972-73 

by Robert E. Klitgaard ' 



AN INTERESTING BUT LIMITED QUESTION 

One part, but only one, of an analysis of an educational program concerns 
students' achievement scores. Although some educators have correctly criticized 
achievement tests as partial and inexact measures of success, there is still a place 
for the assessment of educational programs using test scores. Parents, educators, 
and the public are interested in knowing whether average test scores have gone up 
or down; whether there is more or less equality among students; whether the scores 
of certain socioeconomic and ethnic groups have risen or fallen; and whether more 
or fewer students have surpassed some threshold of basic skills. So long as one 
recognizes that test scores are not the whole story, and are imprecise even in meas- 
uring cognitive skills, the use of achievement tests to analyze program success is of 
interest and importance. 

The Education Voucher Demonstration in Alum Rock is not one but many 
educational programs. Within the six schools participating in the first year, 22 
mini-schools were initiated, differing in curricula, pedagogical methods, classroom 
organization, and educational objectives. A thorough analysis of the demonstration's 
eflect on student achievement would differentiate among the various programs and 
would examine the many effects on scores that programs might have (for example, 
those mentioned in the preceding paragraph).^ This appendix, however, investigates 
only a single broad question: Did voucher programs as a whole tend to raise or lower 
the average achievement score in a grade? 

There are two reasons for limiting the scope of this investigation so severely. 
First, because data on test scores are available only tor the first year of the demon- 
stration, only the broadest trends can be gauged; and even then there is no assurance 
that those trends will persist. (**Learning curves" or ''Hawthorne effects" could be 
invoked to explain away either drops or gains,) Second, intraschool information is 
not currently available; as a result, the effects of different mini-schools within the 
same school and grade cannot be disentangled at present. 

Given these factors, this appendix is confined to comparisons of the average test 
scores of the six voucher schools (for grades 1, 2, 3, and 6) with previous scores from 
those schools, with scores from nonvoucher schools in Alum Rock, and with the 
scores predicted by time trends and socioeconomic variables. 



THE DATA 

Every spring, California tests its first, second, and third graders and gathers 
* I am grateful to Kathleen Styles for research assistance. 

" See R. E. Klitgaard. Achievement Scores and Educational Objectives, The Rand Corporation. R-1217- 
NIE, January 1974. 
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information about the number of'students in each public school whose families are 
on welfare (Aid to Families with Dependent Children). Every fall, the state tests its 
sixth and twelfth graders and uathers data on the ethnic composition of'each school 
and about the number of* students eligible for free school lunches. Unfortunately 
(from the analyst's point of* view) California has not been uniform over time in the 
type of tests given, and did not collect AFDC and free lunch information before 
1970-71. Table Vfll-B.l summarizes the data available from the state government. 



Table VIII-B.l 

TEST AND SOCIOECONOMIC DATA AVAILABLE 
FROM THE STATE UF CALIFORNIA 



Item 


1969-70 


19 70-71 


197l-:^2 


1972-73 


19 7 3- 7 A 


First grade tests 
Cooperative Primary 

Reading Test 
State Reading Test 


X 


X 


X 


X 


(a) 


Second grade tests 

Stanford Reading Test 
Cooperative Primary 

Reading Test 
State Reading Test 


X 


X 


X 


X 


(a) 


Third grade tests 

Stanford Reading Test 
Cooperative Primary 

Reading Test 
State Reading Test 


X 


X 


X 


X 


(a) 


Sixth grade tests 

Comprehensive Test of 

Basic Skills 
Lorge-Thorndlke 

Intelligence Test 


X 
X 


X 
X 


X 
X 


X 
X 


X 
no 


Socioeconomic status 
(by jschool) 
% Mexican-American 
% black 
% welfare 
% free lunch 


X 
X 
no 
no 


X 
X 
X 
X 


X 
X 
X 
X 


X 
X 
X 
X 


X 
X 

(a) 

X 



Not yet processed. 




Two comments are in order. First, note the limited socioeconomic information 
that is obtained. It includes only the broadest proxies for the socioeconomic back- 
grounds of a school's students; it is available by school but not by grade or by student; 
and it is not uniform over time. Stem-and-leaf diagrams comparing the percentages 
of students in each school eligible for free lunches over the past three years show 
a strong shift upv^ards, leading one to believe that the definition of eligibility was 
changed: 
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0 5,6,9,9 

1 2,^,4.^,5.6,6,6,7,7,9.9 



4 
5 
6 
7 



2 0,3,4,8 

3 4,9,9 



0 

1 9 

2 4 ,3.7, 7.8,9 

3 2,9 

4 5,8,8,9,9 

5 0,2,3,3,4,7,9 

6 0,7,8 
7 



0 
1 

2 3,8,9 

3 3,4,5 

4 1,8 

5 5,9 

6 0,2,4,5,6,6,9,9 

7 0,0,0,3,5,8 



1970-71 



1971-72 



197^-73 



(Numbers to the left of the vertical line are in units often; to the right in units of 
one; thus, beginning in 1970-71, one reads: ^'5,6,9,9,12,14,14,14'' and so forth/M Also, 
the Lorge-Thorndike intelligence test was not given in Alum Rock in 1973-74. Sec- 
ond, since different tests were used in second and third grades, raw scores for 
different years are difficult to compare; and it is hard to compare raw scores across 
grades. To overcome these problems, scores were converted into percentile scores, 
based on the distribution of California student scores in each grade and year, and 
then they were transformed into standard normal scores with a mean of 0 and a 
standard deviation of 1. 

The spread of scores among students is of course wider than the spread of school 
or district mean scores. Thus, for example, if a school had an average raw score of 
20 on the first grade Cooperative Primary Reading Test, this corresponded to the 
16th percentile among school averages, but the 31st percentile in interstudent 
terms. The interstudent percentile was chosen as the form of reexpression because 
it was available over all years and grades. 

In transforming scores into standard normal scores the 20th percentile, for 
example, is a standard normal score of —0.842, the 50th percentile a score of 0, and 
the 90th percentile a score of 1.282. Percentile scores are difficult to work with 
statistically because a five-point increase signifies a much greater increase when it 
occurs in the tails than it does around the mean. Using the standard normal score 
overcomes this problem. 



PERFORMANCE OF VOUCHER SCHOOLS BEFORE AND 
AFTER 

Table VIII-B.2 displays the 1971-72 reading scores of voucher schools with those 
a year later, after the voucher demonstration had begun. For the sixth grade, the 
1973-74 scores are also shown. 

In the first, second, and third grades, 14 of the 15 scores showed drops. (The only 
gainer was Cassell in the first grade, a rise of 0.6 points.) A Wilcoxon signed rank 
test indicated that the difference between the two years was statistically significant 
at the 0.062 level for the first grade and the 0.031 level for second and third. The 
average loss was 2.3 points in the first grade, 3.3 in the second grade, and 1.6 in the 
third. (A Wilcoxon signed rank test was used for several reasons. It would not be wise 
to assume that the two years' scores represent independent samplings from normal 
distributions, because the distribution across voucher schools of each year's scores 
is not normal-looking, or even symmetrical. Also, the sample size is very small. As 



Se'* John W. Tukey. Exploratory fkita Analysis, limited preliminary ed.. Addison -Wesley, Reading. 
Mass.. 1970. 
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Table VIll-B.2 
IlAW IlEADIN't: SCORKS IN VoUCHLR SCHOOLS 



School 


Firyt 


Grade 


Second 


Grade 


'Iliird 


Grade 




Sixth Grad 


e 


1971-72 


1972-73 


1971-72 


1972-73 


1971-72 


1972- 


-73 


1971-72 


1972-73 


1973-74 


Cassell 


18.0 


18.7 


20.8 


19.7 


28.9 


28 


3 


(a) 


42.9 


42.5 


Goss 


24.3 


20.5 


30. 1 


22.3 


32.4 


29 


1 


43.6 


40.2 


41.3 


McCollam 


20.6 


19.5 


24.5 


21.7 


34.8 


31 


6 


(a) 


(a) 


(a) 


Meyer 


27.2 


24.4 


25.2 


24.5 


31.5 


31 


0 


(a) 


44.2 


39.2 


Miller 


24.7 


20.5 


25.6 


21.4 


27.9 


27 


.6 


42.2 


38.5 


39.9 


Pala 
















52.3 


50.5 


47.4 


Dorsa^ 
















(a) 


(a) 


36.0 


Fischer^ 
















43.9 


44.7 


42.8 


Sheppard^ 
















44.9 


41.8 


43.3 



^No sixth grade in that year. 

^Nonvoucher school in 1972-73, but became a voucher school in 1973-74. 



a result, a t-test seems unwise. The Wilcoxon test enables us to take advantage of * 
the matched pairs of observations, without assuming normality and without giving 
*'undue" weight to ^'outliers." For the first grade, the Wilcoxon statistic W = 13; for 
the second and third, W = 15; the probability that W would be that large by chance 
is, respectively, 0.062 and 0.031.) 

The sixth grade scores are more difficult to interpret. The fall 1972-73 tests were 
given after only two months of the voucher demonstration; it would be difficult to 
assess a program after such a short time. The three schools for which comparisons 
with the previous year can be made all had lower scores, averaging 3.0 points less. 
If, instead, one compares the fall 1973-74 results with the scores in the same schools 
two years earlier, once again all schools have shown losses, the average drop being 
2.4 points (P - 0.031). 

Perhaps, however, 1971-72 was simply a very good year for the voucher schools 
and the lower 1972-73 scores are a return to previous levels; or perhaps the losses 
observed over that one-year span are part of a longer-term trend of lower scores. In 
either case the voucher demonstration could not be blamed for the drop. To test 
these hypotheses. Table VIII-B.3 compares the first year of vouchers with the three 
previous years, giving the mean standard normal score of all voucher schools in each 
cell. (A similar table using trimeans instead of means as the summary statistic in 
each cell yielded qualitatively equivalent results.) 



Table VIII-B.3 

COMPARISON OF NORMALIZED SCORES 
OVER TIME IN VOUCHER SCHOOLS 



Year 


Grade 


1 


2 


3 


6 


1969-70 


-0.50 


-0.37 


-0.61 


-0.57 


1970-71 


-0.24 


-0.23 


-0.53 


-0.69 


1971-72 


-0,15 


-0,25- 


-0.60 


-0.58 


1972-73 


-0.42 


-0.62 


-0.77 


-0.71 


1973-74 








-0.77 
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Except for the first grade, scores were lower in each grade during the first year 
of the voucher demonstration than in any of the previous years. (The 1969-70 first 
grade mean was 0.08 of a standard deviation lower than the 1972-73 average score.) 
There is no significant trend over time in any grade. One does see substantial 
variation in the scores, however. How large are the losses in 1972-73 in terms of this 
variation? 

Table VIII-B.4 shows the results of a two-way analysis of variance of the data 
in Table VIII-B.3. (The 1973-74 score has been inserted for the sixth grade in place 
of the 1972-73 score. State percentiles have not yet been published, so I used the 
1972-73 percentiles to transform the 1973-74 score. Percentiles do not change much 
from year to year, so this adjustment should cause virtually no inaccuracy.) The 



Table VIII-B.4 

TWO-WAY ANALYSIS OF VARIANCE, VOUCHER SCHOOLS, 
1969-70 TO 1972-73 



Year • 


Grade 


Row Effects 


1 


2 


3 


6 


196S-70 


-0.15 


0.02 


0,04 


0,10 


-0.02 


1970-71 


0.02 


0.07 


0.03 


-0.11 


0,07 


1971-72 


0.08 


0.02 


-0.07 


-0.03 


O.iO 


1972-73 


0.06 


-0.10 


0.01 


0.03 


-0.15^ 


Column ef fe - ts 


0.17 


0.13 


-0.13 


-0.16 


-0.43 Grand mean 



Sixth grade score for 1972-73 is actually the score from 
the fall 1973 testing. 



1972-73 'Vow effect*' is -0.15 of an interstudent standard deviation and is larger 
than the effect of any other year ("row"). The row effects are significantly different 
from zero (P = 0.03). These results were reinforced by an analysis of the same table 
using resistant "two-way" techniques developed by Tukey.'* The best guess of the 
effect of the voucher program year, given the performance of the voucher schools 
in previous years, is a drop of about one-sixth of an interstudent standard deviation 
on reading scores. A two-way midmean analysis of the data of Table VIII-B.3 yielded 
the following: 



12 3 6 Row Effects 



1969-70 


-0.25 


-0.04 


-0.01 


0.05 


0.01 


1970-71 


-0.05 


0.03 


0.01 


-0.14 


0.07 


1971-72 


0.03 


0,00 


-0.07 


-0.04 


0.08 


1972-73 


0.00 


-0.13 


0.00 


0.02 


-0.16 


Column effects 


0.20 


0.12 


-0.15 


-0.17 





' Ibid. 
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The cell residuals Ry were regressed against their expected values (the product of 
the column effect i and the row effect j divided by the grand mean), and transformed 
according to the slope of the regression line. In this case, the slope was — 1.2, which 
calls for squaring the original data in order to assess the "true" row effect (see 
Tukey, Vol. III). The final two-way midmean table, afler readjusting signs, was: 



12 3 6 Row Effects 

1969- 70 -0.19 -0.01* 0.00 0.07 0.03 

1970- 71 -0.04 0.03 0.05 -0.12 0.07 

1971- 72 -0.01 0.02 -0.04 0.01 0.08 

1972- 73 0.09 -0.05 -0.02 0.01 -0.18 

Column effects 0.18 0.11 -0.13 -0.16 



In interdistrictstandard deviations, this row effect can be reexpressed as follows: 
if the entire Alum Rock district performed as the voucher schools did, the district 
would have dropped about one-third of a standard deviation compared with other 
districts in California. (This result is based on tables provided by the State of Cali- 
fornia,) 



COMPARING VOUCHER AND NONVOUCHER SCHOOLS 

Perhaps the drop in reading scores observed in the voucher schools was due to 
some more general drop throughout the Alum Rock district. This hypothesis would 
attribute the losses in voucher lools not to the voucher demonstration but to other 
factors operating throughout ihe district. 

Figure VIII-B.l displays stcm-a/id-leaf diagrams for all Alum Rock schools by 
grade, in 1971-72 and in 1972-73. (The units are tenths of an interstudent standard 
deviation.) The scores of voucher schools are circled. In this figure one can roughly 
assess the relative and absolute shifts of voucher and nonvoucher schools over that 
one-year span. (This disparity in the number of schools having sixth grades reflects 
the expansion of two schools and the opening of one new one.) In every case, the 
voucher schools have dropped relative to the nonvoucher schools in Alum Rock. 

How have the nonvoucher schools performed over time? Table VIII-B.5 presents 
the parallel to Table VIII-B.3 for the nonvoucher schools. There appears to be no 
time trend in any grade. Interestingly, the 1972-73 row effect is slightly positive, as 
Table VIII-B.6, showing the two-way analysis of variance of the data in Table VIII- 
B.5, indicates. Compare these results with those in Table VIII-B.4. Again, two-way 
midmean analysis reinforced this finding. The first iteration yielded: 





1 


2 


1969-70 


-0.02 


U,03 


1970-71 


-0.05 


0,04 


1971-72 


0.04 


-0.04 


1972-73 


0.18 


0.01 


Column effects 


0.10 


0.05 



3 6 Row Effects 



0.14 


-0.07 


0.04 


0.01 


0.05 


-0.01 


-0.01 


0.10 


-0.03 


-0.01 


-0.03 


0.01 


-0.11 


-0.04 


'0.47 
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Table VIII-B,5 

NORMALIZED SCORES OVER TIME FOR 
ALUM ROCK NONVOUCHER SCHOOLS 



Year 


Grade 


1 


2 


3 


6 


1969-70 


-0,36 


-0,36 


-0,41 


-0,55 


1970-71 


-0,44 


-0,40 


-0,59 


-0.48 


1971-72 


-0,36 


-0,49 


-0,62 


-0,44 


1972-73 


-0,19 


-0,41 


-0,59 


-0.50 


1973-74 








-0,54 



Table VIII-B,6 

TWO-WAY ANALYSIS OF VARIANCE, NONVOUCHER SCHOOLS 
IN ALUM ROCK, 1969-70 TO 1972-73 



Year 


Grade 


Row Effects 


1 


2 


3 


6 


1969-70 


-0,05 


0,02 


",11 


-0,08 


0,03 


1970-71 


-0,08 


0,04 


,01 


0,05 


-0,03 


1971-72 


0,00 


-0,05 


-0,04 


0,09 


-0,03 


1972-73 


0,13 


-0,01 


-0.06 


-0,06 


0 . 02 


Column effects 


0,11 


0,04 


-0,10 


-0,05 


-6\ 4b 



NOTE: Sixth grade score for 1972-73 is actually the 
score from the fall 1973 testing. 



The regression of cell residuals against plotting values yielded a slope of -4.7, 
calling for a transformation of the original data to squares. This done, the final 
two-way midmean analysis, with signs corrected, was: 



1969-70 


-0 


02 


0 


02 


0,15 


-0 


07 


0,03 


1970-71 


-0 


05 


0 


02 


0,01 


0 


04 


-0,01 


1971-72 


0 


04 


-0 


03 


-0,01 


0 


10 


-0,03 


1972-73 


0 


09 


-0 


01 


-0,01 


-0 


04 


0,01 


Column effects 


0 


09 


0 


06 


-0,12 


-0 


03 





There was, then, virtually no overall change in 1972-73 from previous reading test 
score averages in the nonvoucher schools. 

Given the overall variation in Table VIII-B.5, however, it is best to conclude that, 
unlike the results for the voucher schools, there is no significant 1972-73 row effect 
for the nonvoucher schools. 

These results lead one to conclude that the drop in achievement scores witnessed 
in the voucher schools cannot be explained by an overall drop in the scores of the 
Alum Rock district as a whole. In the five Title I schools in 1972-73, there was 
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apparently no significant overall gain or loss in reading scores. Of the fifteen first, 
secoiul, and third grade scores, nine were higher, five lower, and one stayed the 
same; on average the first grade scores dropped slightly, and the second and third 
grade scores went up. The Title I schools as a whole had lower scores than the 
voucher school? in 1971-72, but after the first year of the demonstration, the voucher 
schools were lower in every grade for which comparisons could be made. (There was 
only one Title I sixth grade in 1972-73.) Here are the Title I school scores: 







1 


2 


3 


6 


Arbuckle 


1970- 71 

1971- 72 

1972- 73 


-0.553 
-0.100 
-0,050 


-0,467 
-0.643 
-0,331 


-0,878 
-0.772 
-0,772 




Hubbard 


1970- 71 

1971- 72 

1972- 73 


-0.025 
0.025 
-0.125 


-0.495 
-0.202 
-0,331 


-0.524 
-0,643 
-0,412 


-0.674 


May fair 


1970- 71 

1971- 72 

1972- 73 


-0,994 
-0,253 
-0,553 


-1,126 
-0.706 
-0,495 


-0.841 
-0.954 
-0.878 




San AnConlo 


1970- 71 

1971- 72 

1972- 73 


0.279 
0.151 
0.025 


+0,772 
-0.643 
-0.915 


-0.467 
-0,674 
-0.524 




Slonaker 


1970- 71 

1971- 72 

1972- 73 


-1,080 
-0.739 
-0,253 


-0,612 
-1,036 
-0.643 


-0.841 
-0,954 
-0.915 





The average scores over all Title I schools for the past three years are: 

12 3 

1970- 71 -0.586 -0.386 -0,710 

1971- 72 -0,183 -0,646 -0.799 

1972- 73 -0,191 -0,543 -0,700 



Grade-equivalent (GE) and raw scores are provided below, so that the 1972-73 per- 
formances of voucher and Title I schools on the Cooperative Primary Reading Test 
can be compared with their scores on the Metropolitan Achievement Test, as given 
in Appendix VIII-C. 





1 




2 




3 




Voucher Schools 


Raw 


GE 


Raw 


GE 


Raw 


GE 


Cassell 


18.7 


1,5 


19,7 


2.1 


28,3 


2.7 


Goss 


20,5 


1,7 


22,3 


2.3 


29,1 


2,8 


McCollam 


19,5 


1.7 


21,7 


2,3 


31.6 


3,2 


Meyer 


24.4 


1.8 


24.5 


2,5 


31,0 


3,0 


Miller 


20.5 


1.7 


21.4 


2,2 


27.6 


2,7 




20,72 


1.7 


21,92 


2,3 


29,52 


2,9 


Title I Schools 














Arbuckle 


23,8 


1,8 


24.3 


2.5 


27,6 


2,7 


Hubbard 


23,0 


1,8 


24,5 


2,5 


33,5 


3,3 


May fair 


19.6 


1,7 


22,9 


2,4 


28,4 


2.7 


San Antonio 


24,6 


1.9 


19,5 


2,1 


32,3 


3.1 


Slonaker 


21,8 


1,8 


21,6 


2.3 


2G,1 


2,7 




22,56 


1,8 


22,58 


2,4 


29,98 


2,9 
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CONTROLLING FOR SOCIOECONOMIC DIFFERENCES 

Other explanations (or the drop in the scores in voucher schools might be sug- 
gested. For example, suppose the introduction of the voucher program was accom- 
panied by changes in the socioeconomic and demographic characteristics of the 
students in voucher schools. If these changes themselves led to lower scores on 
reading tests, then the voucher demonstration could not legitimately be held respon- 
sible for the losses. 

Unfortunately, the California data provide little in the way of socioeconomic 
information. Along the variables that one can analyze, however, there seems to have 
been no important change between 1971-72 and 1972-73 for the voucher schools, 
either absolutely or relative to the other schools in the Alum Rock district. Figure 
VIII-B.2 provides a gross but useful depiction of the changes along three variables 
of interest. (The percentage ofstudents receiving free lunches did increase over that 
year, as previously noted; but relative positions of voucher and non voucher schools 
did not change significantly.) 

Rand's analysis of the long-term trends in socioeconomic variables in Alum Rock 
has uncovered no significant departures during 1972-73. (The analysis will be 
presented in a forthcoming technical report) 

As a further examination of the influence of socioeconomic variables on achieve- 
ment scores, I ran a large number of simple and multiple regressions using the 
limited available Alum Rock data. I checked the explanatory power of the various 
socioeconomic proxies, experimenting with many transformations and combina- 
tions, stratifying the data on some occasions and combining years on others, omit- 
ting and including apparent outliers, and using ordinary least squares as well as 
more resistant fitting techniques. My results can be summarized as follows: 

The socioeconomic proxies have very limited predictive power for mean reading 
scores in grades /, 2. and 3 in Alum Rock. For the first grade, I could not obtain a 
reasonable regression line; the only one that cam.e close to statistical significance 
loaded the percentage ofstudents eligible for free lunches with a positive coefficient 
(perhaps because of the apparent definitional shifts in that variable over time), and 
even then under 8 percent of the total variation was explained. 

For the second grade, my exertions were again unsuccessful in achieving a fit. 
The best regression attained a t-statistic of 1,3 (barely significant at the 0.10 level) 
and explained only about 3 percent of the variation in achievement scores. 

For the third grade, the best equation was 

Y = 0.02 - 0.13 (In % black) - 0.10 (In % AFDC), 
(0.04) (0.06) 

where Y is the mean achievement score for the third grade (in Alum Rock schools 
from 1970-71 through 1972-73, excluding the five voucher schools in 1972-73 only), 
and the figures beneath the regression coeflScient are the standard errors of the 
coefficients. R^ = 0.35; standard error of the dependent variable = 0.18; N = 51. 
One outlying observation was omitted to calculate this fit. 

Good predictions could be made of past sixth grade scores, but data limitations 
precluded detailed analysis. Using transformations of the Lorge-Thorndike IQ test 
and the ethnic variables, I could explain about 92 percent of the variance in grade 
6 mean scores, 1969-70 through 1972-73, However, the Lorge-Thorndike test corre- 
lates so highly statewide with the sixth grade reading scores (p = 0.92) that one 
suspects it is merely another sort of achievement test. Furthermore, the Lorge- 
Thorndike test was not given in Alum Rock in the fall of 1973-74. One is left with 
a questionable prediction equation that is useful only to analyze reading scores that 
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Fig. VIII-B.2—Stem-and-leaf diagrams of socioeconomic proxies 
for Alum Rock schools, 1971-72 and 1972-73 
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were obtained afler two months of the voucher demonstration (i.e., fall 1972-73). For 
what it is worth, the voucher schools in 1972-73 performed worse than the non vouch- 
er schools in the same year, compared with what would be expected on the basis of 
previous years a/'/trcontroUing for IQ and the natural logarithm of the percentage 
of black students. Ranking the residuals in order, with eleven nonvoucher and five 
voucher schools, resulted in the following picture (where X's indicate voucher 
schools): 

Rank 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 
School XOOOOOOOOOOXOXXX 

The probability of such a ranking occurring by chance in two such samples from the 
same distribution can be gauged using a Mann-Whitney test, the result being 

z ^ - ~ MT ^ 58 - V^ - 42.5 ^ 

o-T 8.83 ' ' 

P (Z > z)< 0.05. 



The average difference between voucher and nonvoucher schools after controlling 
amounted, in grade-equivalent terms, to about two months. However, given the 
shortcomings mentioned earlier, it is probably best to discount these findings. 

One can also make fairly good predictions of sixth grade scores using only the 
socioeconomic proxies (R^ > 0.6), but the equation must make use of the AFDC 
percentage. Since this statistic is not yet available for the 1973-74 school year, such 
an equation is fruitless in analyzing the more recent (and more meaningful) sixth 
grade scores. 

Regression results were consistent with the earlier picture of a drop of about 0.15 
to 0.20 of an interstudent standard deviation in the voucher schools during 1972-73. 
For example, a dummy variable was introduced for the voucher schools during the 
voucher years. The coefficients were -0.12 for the first grade, -0.23 for the second, 
and -0.15 for the third (after controlling for In % black and In % AFDC), and 
despite the small number of voucher observations and the nonmatched samples, the 
t-statistics reached 0.8, 1.6, and 1.6, respectively. 

More directly. Table VIII-B.7 compares the residual scores (actual minus pre- 
dicted) over time for each voucher school. A positive score indicates that the school 
performed above expectation, a negative score that it fell below. (The results for 
grades 1, 2, and 3 only are provided because of the limitations described above on 
the sixth grade data.) Except for CasselTs first grade, 1972-73 showed a drop in 
reading score residuals in all grades in all voucher schools. Table VIII-B.8 provides 
the average voucher school scores over time, and Table VIII-B.9 shows the results 
of a two-way analysis of variance of these data. Once again, the result is that the 
voucher schools dropped during 1972-73. Row effects are nonzero at the 0.003 level. 
Two-way midmean analysis again yielded similar results: 



1 

1970- 71 0.00 

1971- 72 0.00 

1972- 73 0.00 



2 3 Row Effects 

0.00 -0.03 0.10 

0.00 0.00 0.08 

-0.10 0.04 -O.J,9 



Column effects 0.01 



0.03 -0.04 
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Table VIII-B,7 
RESIDUAL SCORES FOR VOUCHER SCHOOLS 







Grade 


School 


Year 


1 


2 


3 


Cassell 


1971 
1972 
1973 


-0.38 
-0,48 
-0,41 


0 

-0.30 
-0,51 


-0,22 
-0,11 
-0.23 


Goss 


1971 
1973 


0,60 
0,34 
-0.12 


0,56 
0,61 
-0.14 


0.34 
0,23 
-0,13 


McCollam 


1971 

1^72 

1973 


-0.07 
-0.09 
-0,20 


0 

0,10 
-0,24 


0.15 
0,43 
-0,01 


Meyer 


1971 
1972 
1973 


0,12 

0,57 
u, 


0,02 
0,18 
0.08 


0,06 
0.11 

0,02 


Miller 


1971 
1972 
1973 


0.57 
0.37 
-0,12 


0,33 
0,21 
-0,24 


0,08 
-0.20 
-0,31 



NOTE: Units are interstudent 
standard deviations. 



Table VIII-B,8 

AVERAGE RESIDUAL SCORES IN VOUCHER 
SCHOOLS: ACTUAL MINUS PREDICTED 
NORMALIZED SCORES 



Year 


Grade 


1 


2 


3 


1970-71 


+0,17 


+0,18 


+0,08 


1971-72 




+0,16 


+0,09 


1972-73 


-0.12 


-0,21 


-0,13 



Table VIII-B,9 

TWO-WAY ANALYSIS OF VARIANCE: VOUCHER SCHOOL 
RESIDUAL SCORES 



Year 


Grade 


Row Effects 


1 


2 


3 


1970-71 


0,00 


0,03 


-0.04 


0.10 


1971-72 


-0,01 


0.02 


"0, 01 


0,09 


1972-73 


0,01 


-0,06 


0.05 


-0,20 


Column effects 


0.02 


0,00 


-0,03 


0,04 



118 



No need was found for further iterations. 

llovv did these results compare with those for the nonvoucher schools? Table 
VIII-B.IO gives the results of a two-way analysis of variance of the average residual 
scores in nonvoucher schools from 1970-71 to 1972-73, grades 1, 2, and 3. In contrast 
to the voucher schools, the nonvoucher schools showed no significant drop in 1972- 
73; in fact, they gained a bit. Once again, the two-way midmean analysis supported 
the finding of the two-way analysis of variance. The corresponding 1972-73 row effect 
was also 0.04. 



Table VIII-B.IO 

TWO-WAY ANALYSIS OF VARIANCE: NONVOUCHER SCHOOL 
RESIDUAL SCORES 



Year 


Grade 


Row Effects 


1 


2 


3 


197C-71 


-0.07 


0.04 


0.03 


-0.02 


1971-72 


U.09 


-0.02 


-0.06 


-0.02 


1972-73 


-0.01 


-0.01 


0.03 




Column effects 


-0.0^ 


0.00 


U.U3 





CONCLUSIONS 

Voucher schools recorded lower average reading scores in every gi ade in 1972-73 
than in previous years, while nonvoucher schools stayed about the same. This result 
held even after controlling for the limited socioeconomic information available on 
Alum Rock schools from the State of California. 

How should one interpret this result? Two quesuuiis arise: How large v/as the 
drop? Hovv b'igni/icani is it from the standpoint of assessing the voucher experiment 
or, more broadly, for educational policy? 

How large were the losses? In terms of interstudent standard deviations, the 
average drop in the reading scores in voucher schools across all grades tested was 
0,15 to 0.20. In interdistrict terms, the drop was about one-third of a standard 
deviation. 

Reexpressing these two equivalent results in percentiles, if a student performed 
at the 50th percentile before and dropped 0.15 to 0.20 of a standard deviation, he 
would end up at the 42nd to 44th percentile. If a district were at the 50th percentile 
in interdistrict terms and fell by a third of a standard deviation, it would be about 
the 37th percentile. 

What about grade equivalents? They are of little use in the first, second, and 
third grades, because there is so little variability in grade equivalent scores and 
because "floor effects" are so important. (For example, the 50th interstudent percen- 
tile on first grade scores in California in 1972-73 was "equal*' to 1.8 in the publisher s 
grade equivalent; but the 25th percentile was only a drop to a score of 1.6.) For what 
it is worth, for the first, second, and third grades, 0.15 to 0.20 of a standard deviation 
is in no case more than two months in grade equivalent terms. 

The second question — the significance of these results for an assessment of the 
voucher demonstration — clearly transcends mere numbers. There is first of all a 
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Vdliie judt^inont as to "how nuich'' such a drop matters. Do decisions about the 
continuation or the desired size ofthe demonstration depend on increased achieve- 
ment scores? Second, there are obvious diiUculties in assessing any new and diver- 
sified program after only one year of operation and using only broad comparisons 
of average scores. More discriminating analyses, afler more time has passed and 
more data have been collected, will of course be necessary to gauge the true effects 
on achievement scores. 



Appendix VIII-C 



PRELIMINARY ANALYSIS OF METROPOLITAN 
ACHIEVEMENT TEST SCORES, VOUCHER 
AND TITLE I SCHOOLS, 1972-73 ' 

by Pierce Barker 



INTRODUCTION 

Cronbach and Furby asserted in 1970 that, given a nonequivalent control group 
design, "What cannot be done is to compare 'treatment effects/" If we take that 
rather dogmatic pronouncement completely seriously, we educational researchers, 
given the present state of the art, must throw in our hands. In most field settings, 
the nonequivalent control group (Campbell and Stanley, 1973) is the best we are 
likely to get; and if we are to believe that differences between ''treatments*' in these 
studies cannot be meaningfully compared, we are wasting our time — we should not 
be doing the research at all. 

The picture need not be as bleak as all that, however; there are situations in 
which a careful assessment of the context of the research may lead to structural 
models in which the "treatment effects" can be compared. This appendix investi- 
gates that possibility with available achievement test data from the Alum Rock 
demonstration. 



THE SETTING AND THE DATA 

In the fall of 1972 and again in the spring of 1973, Metropolitan Achievement 
Tests (Harcourt et al., 1971) were administered to five elementary schools designated 
by the Alum Rock district as Title I schools, and five others in the same district that 
were participating in the voucher demonstration. The testing was done independent- 
ly in the two groups of schools, for purposes of evaluation of the two progams; the 
spring testing was done in the two groups roughly contemporaneously, while in the 
fall, the voucher schools were tested six weeks later than the Title I schools. 

One clear difference in the testing situations must be described, although its 
effect cannot be estimated. In the Title I schools, the test designated by the publ isher 
on the basis of national norms as appropriate for any grade (k) was administered to 
all students in that grade; that is, all testing was "in-level." In the voucher schools, 
however, teachers were permitted to give to each student a test designed for the 
grade at which, in the teacher's judgment, the student was actually achieving or was 
capable of handling. The result was that it was not unusual for students enrolled 
in a grade (k) to be given tests designed for grades (k + 2). In order to compare 
student's m.arks on these tests among themselves or across grades or groups, it was 
necessary to convert the raw scores to a common base. In the case of the MAT, this 

' The author gratefully acknowledges the advice and assistance of Professor Finis Welch of the 
Economics Department. UCLA, and The Rand Corporation. He managed to convert a piece of panicky 
drudgery into an occasion ofiearning; but he is not, of course, in any way responsible for the use made 
of his advice, nor for errors in this piece. 
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common base is the Grade Equivalent iGE) scale. It is not clear, howeverv, that the 
conversion algorithm worked out by the publisher on a national sample is suitable 
for a sample consii-'.ting largely of low-achieving students. On the basis of some 
preliminary data gathered to assess the accuracy of the conversion for Alum Rock 
studeriis, it is possible to say that the standard conversion algorithm produced 
aggregate scores for a set of third grade students, given both the third grade and 
second gr ide tests in randomly counterbalanced order, which were accurate to about 
0.2 GE units or less (95 percent confidence limits). (This study is described in full in 
Barker, forthcoming.) Consequently, while this problem does not seem on the basis 
of preliminary analysis to be severe, it should be borne in mind in considering the 
results of the present analysis. 

Since most voucher first graders were given the lowest level of the MAT (the 
Primer), which does not translate to GE units, only the second through fiflh grade 
results are included here. The data for Title I schools were available only in con- 
densed form; consequently, all analysis is based on school means by grade. Since the 
res:i'lts for reading and math are very similar, only the reading scores are considered 
here. 

The basic data matrix upon which all subsequent analysis is based is shown in 
Table VIII-C,1, 



STATEMENT OF THE PROBLEM 

The Title I program, like a great many state and federally-funded educational 
programs, is compensatory; it is intended to aid disadvantaged students to improve 
their level of academic achievement. Title I students, in school districts which opt 
to designate individuals, are selected on the basis of an economic indicator of relative 
poverty. Districts may, however, choose to designate entire schools as Title I schools, 
and the criterion again is economic; specifically, schools are chosen based upon the 
proportion of students in a school who are from families on welfare. Operationally, 
designation ofTitle I status is based on an annual census of students who are eligible 
for AFDC (Aid to Families with Dependent Children); Title I schools in a ^y district 
typically are those for which the proportion of AFDC recipients is highest That is, 
selection as a Title I school is a consequence of a ^rowp characteristic, not a charac- 
teristic of individual students; moreover, selection is not explicitly based on individu- 
al or group academic achievem.ent, however measured. It is, of course, commonplace 
that socioeconomic status (SES), of which eligibility for AFDC is considered a valid 
indicator or proxy, is negatively correlated with academic achievement; still, we 
emphasize that achievement per se is not the basis for selection. 

Now, the central problem of causal inference in nonexperimental settings, par- 
ticularly for the nonequivalent control group design, is precisely the composition of 
the groups by a process of deliberate selection. In a randomized ("true*') experimen- 
tal design, however complex, the central justification for causal inference from 
group differences (^'treatments") to observed effects on the dependent variable(s) is 
that, by design, differences among group other than the treatments are correlated 
with the treatments only by chance, 

"In a fully randomized experiment, the covariates have only a small chance 
correlation with the experimental treatments and hence the expectation of 
the statistical [covari'ance] adjustment is zero," (O'Connor, 1973, p, 4) 

More specifically, in a pre-posttest design, the expected value of the correlation 
between the treatment and the pretest is zero; and this is also true of the correlation 



Wibie VUI-C.l 

MEAN PRl- rESl /VND POSTTi-S'l KKADINC SCORKS FOR VOUCHER 
.\ND TlTLfc: I SCHOOLS, BY GRADE LKVEL 



u rade^ 




p ■> 1 1 
r ai i 


1972 




Spring 1973 


Gain 


V 


11 




11 


V 


n 


T 


n 


V 


T 




2.15 


51 


1 


D 2 


1 HQ 

1 U 7 


2 . 59 


bO 


2 . 06 


109 


0.44 


0 , 54 




1 9 7 


40 


1 


56 




2 . 60 






80 


0.63 


0.70 




2 . 08 


60 


1 


48 


1 1 9 
LLC 


2 . 80 


56 


2.11 


111 
112 


0.72 


0,63 




2 11 


SQ 


1 


-?4 


O J 


2,54 


64 


2.24 


63 


0.43 


0.67 




•- . uo 


32 


1 


50 


8 7 


2.43 


40 


2.20 


87 


0. 37 


0. 70 


Mean 


2.08 


242 


1 


52 


451 


2 , 59 


2 52 


2 . 16 


451 


0.51 


0 . 64 




HQ 

J • U7 


90 




2 3 


108 


3 . 59 


102 


2.73 


108 


0, 50 


0, 50 




^ SQ 


55 


2 


26 


7U 


2 , 88 


68 


3.17 


90 


0.29 


0.91 




> '\f\ 

i. . .>o 


h 1 


2 


14 




3 . 06 


85 


2 . 52 


95 


0 . 70 


0, 38 




2 . 57 


87 




12 


49 


3,07 


89 


3.10 


49 


0, 50 


0,98 




2.40 


60 


2 


.07 


80 


3,07 


48 


2.53 


80 


0.67 


0,46 


Mean 


2*.'64 


'353 


2 


17 


422 


3.17 


392 


2.78 


422 


0,53 


0,61 




3.38 


74 


2 


50 


99 


3.80 


76 


2,95 


99 


0.42 


0,45 




2,83 


62 


T 


56 


91 


3,24 


70 


3.24 


91 


0.41 


0.68 


4 


2.91 


97 


2 


48 


100 


3.51 


106 


2.85 


100 


0,61 


0.37 




3.27 


71 


2 


47 


50 


3,45 


82 


3.10 


50 


0,28 


0.53 




2.77 


51 


2 


46 


71 


3,13 


42 


3,30 


71 


0.36 


0.84 


Mean 


3.05 


355 


2 


50 


411 


3.56 


376 


3.07 


411 


0.51 


0,57 




4.37 


83 


3 


11 


80 


5.05 


103 


3.68 


80 


0.68 


0.59 




3.64 


75 


3, 


26 


69 


3.92 


81 


3,68 


69 


0,28 


0.42 




3.55 


98 


3 


22 


91 


4.17 


104 


3,57 


91 


0,62 


0.35 




3.77 


87 


2 


80 


40 


4,25 


96 


3.44 


40 


0,48 


0.64 




3.67 


51 


3 


40 


57 


4. 12 


52 


4.20 


57 


0.45 


0.80 


Mean 


3,80 


394 


3 


18 


337 


4, 34 


4 36 


3.71 


337 


0.54 


0,53 



NOTE: V = voucher schools, T = Title I schools. Vouclier 
school grades have been translated Into grade equivalent 
units. 



"^The voucher schools (McCollam, Goss, Meyer, Cassell, 
Miller) and Title I schools (San Antonio, Hubbard, Slonaker, 
Mayfair, Arbuckle) appear in that order for each grade in 
this table, 

^All grade means are *>: Jr- ' jhi-.'-i means of school means; 
they are based, not upon individual scores, but upon ;// 
scores, A clieck of several sets of scores at random indi- 
cates that this procedure will not differ by more than about 
0.02 f ro:a the weighted mean based on individual scores. 

between the treatment and^ any other concomitant variable, measured or un- 
measured. The chance correlation is fully allowed for in the test of statistical signifi- 
cance; and in those occasional instances when randomization does produce a large 
correlation between treatment and some measured variable, including the pretest, 
statistical adjustment by analysis of covariance is perfectly proper, provided the 
data meet the assumptions underlying the covariance model (Cochran, 1957; Ela- 
shoff, 1969; Kenny, 1974). 

However, selection for compensatory treatment, on whatever basis made, repre- 
sents a deliberate, nonchance correlation of some individual and/or group charac- 
teristic(s) with the treatments. Specifically, the compensatory educational Title I 
programs deliberately enroll students, as we have seen, on the basis of a characteris- 
tic firmly believed to be correlated with academic achievement; and academic 
achievement is usually the principal dependent variable in assessments of the 



123 



"effects of the treatment/' Said differently, the "treatment'' and "control" groups 
in these programs differ by design on the dependent variable and on the selection 
variable, so that the "treatment'' and the selection variable are deliberately highly 
correlated: "treatment" and one or more other characteristics of the groups are 
deliberately confounded. And it is not just the selection characteristic(s), but other, 
usually unmeasured characteristics, which are also confounded with the treatment. 
But this means that, in simplest terms, any difference(s) on the posttest(s) between 
the two groups, which the researcher would like to attribute to the "treatment," may 
usually plausibly be attributed to nontreatrnent differences; these are the "plausible 
rival hypotheses," which Campbell and Stanley (1963) rightly and vividly portray 
as the betes noirsoi non- or quasi-experimental research. 

Researchers in education refer to this as the problem of confounding; statisti- 
cians and economists (e.g., Goldberger, 1964) are more likely to speak of specification 
error. The problem is clearly and simply stated by Werts and Linn (1970) as follows: 

. . . if a variable is not included in the regression equation then the assump- 
tion is that either it is not an influence on the dependent variable or, if it 
is an influence, it is iincorrelated with all the independent variables in the 
equation. Failure to include relevant influences then can in genera! be ex- 
pected to result in biased estimates for a/^the variables studied (p. 18; italics 
added). 

A very clear hypothetical example of this sort of bias appears in O'Connor (1973). 
Campbell and Erlebacher (1971) make an eloquent suggestive case for the existence 
of specification bias in evaluation of compensatory education programs; Kenny 
(1974) documents the existence and assesses the strength and direction of specifica- 
tion bias in real data. In passing, this discussion clarifies the value of random 
assignment to treatment groups; randomization guarantees within the limits of 
chance that variables not included in the analysis are uncorrelated with the ''varia- 
bles studied," in particular the treatment variable. Hence, omission of explicit 
consideration of other variables may aflfect the sensitivity o{ the analysis to detect 
treatment effects, but will nothxas the estimates of these eflfects. 

The problem, then, in evaluating the effects of, say, compensatory educational 
programs designed for deliberately selected individuals, is simply, how can we cor- 
rectly specify our analytic model, or, how can we ensure, by including explicitly 
correlated influences, that our estimates of treatment effects are not biased? As we 
have seen, many skilled and knowledgeable (and influential) researchers say that it 
simply cannot be done (e.g., Cronbach and Furby, 1970; Lord, 1967). 



ONE DISSENTING OPINION AND A MODEL 

Implicit in the work of, for example, Goldberger (1972 (a), (b)) is the notion that, 
for a given structural model, the existence of bias depends very much upon the 
criterion or manner of selection of individuals or groups into the "treatment" and 
"control" (better, comparison'') groups. That is, given that researchers apply one 
specific model in every situation, their efl[ects estimates will or will not be biased, 
depending upon the fit of the model to the research context. Put thus baldly, this 
has the air of tautology; but, given the known and obvious propensity of researchers 
in this field in factto use a single model (typically, a covariance model), Goldberger's 
work is very much to the point. 

In a paper soon to be published, Kenny (1974) carries this work a large step 
forward by proposing a general model of selection and, for several popular analytic 
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models, specifying the /?ro/?cT model in terms of the particular criteria by which 
solections in given situations are made. Briefly, he proposes a mode! in which the 
pretest and posttest are causal functions of group, individual, and random variables; 
and in which the A^m^/ of confounding of treatment with selection depends upon 
selection in terms of one or another of these variables. His expository strategy 
involves, for the popular analytic models he considers, a determination of which of 
the null hypotheses goes to zero in the absence of treatment effects. His general 
model assumes that the posttest is a function of the pretest and the treatment (cf. 
Werts anH T inn, 1970), and that these in turn are functions of group differences (by 
hypothesis, perfectly stable over time), individual differences (correlated over time 
but not perfectly stable), and "totally unstable causes . . . (errors of measurement).'' 
His model, then, may be viewed as (with variables for convenience in standard form) 

A 

X2 = /^21.tXi + 2T.iXt , 

where = score on the posttest 

Xi score on the pretest 

Xt = treatment 

^21.T = (ri2 - riTr2T)/(i - ^\T^) 

^2'r.i = (r2T - riTri2)/(i - rir^) . 

And these variables in turn may be expressed as 

X2 = flG, Zi, El) 
Xi = fiG, Z2, E2) 
Xt = flG, Zi, Ei, Ui) , 

where G = stable group characteristics 

Z = Correlated (over time) individual characteristics 

E = error of measurement 

U = other causes of selection than G, Z, or E. 

The assumption is that 

Gov (Qi, Ri) = 0, Q, R ^ Z. 

Then it is easy to derive, from these structural equations, expressions for the inter- 
correlations of the pretest, posttest, and treatment, and hence to write the expres- 
sions for the betas. 

Now by specifying various selection patterns in terms of G, Z, and E, by permit- 
ting the coefficients of one or more of them to equal zero, and setting a "no treatment 
effect" condition by setting 

and setting other conditions on P'li^Ty he derives expressions for the null hypotheses 
of the various popular models which he considers in terms of equalities of various 
functions of the intercorrelations of the pretest, posttest, and treatment of the 
reliability of the pretest, and determines, for each selection pattern, which equality 
holds, thereby determining the proper analytic model for specified modes of selec- 
tion. 

Simply stated, the strategy is this: (1) Given a particular model, say a covariance 
model, the null hypothesis of no treatment effect yields certain relationships among 
the mode parameters, and (2) also, given a structural model in terms of selection 
criteria, which also assumes no treatment effect, allowing selection to vary by cri- 
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teria also yields relationships amonf; the model parameter. The proper model for 
analysis is chosen by determining a match between the deductions from (1) and from 
(2). 

This is a very meager precis of his paper, and does not begin to do justice to the 
breadth and incisiveness of his treatment. Moreover, for the record, he writes his 
regression equation in terms of covariances, not correlations; I have chosen to re- 
state it in correlational terms with the feeling that more readers will be familiar 
with this formulation. 

Now, the situation in which the data under consideration were generated seems 
to provide a rather straightforward and clear example of one of the modes of selec- 
tion which he considers, i.e., selection on the basis of^ group characteristics. 

Without going into the details of the derivation, we will simply state that selec- 
tion on the basis of group characteristics, given the model with which we ar^^dealing, 
indicates that the proper test of the null hypothesis is a test of the equality 

r2T = riT , 

i.e., that the pretest-treatment correlation equals the posttest-treatment correla- 
tion." The model is stated in terms of two groups, a '^treatment'' and a ''control" 
group, so that the treatment variable Xt is a binary variable (usually coded 1/0) and 
the correlation of the treatment variable with the (presumably) continuous criterion 
variables are point-biserial correlations (McNemar, 1969; Nunnally, 1967). 



TEST OF THE SELECTION MODEL ON TITLE I/VOUCHER 
DATA 

The data we are dealing with fit the version of the stated model in that selection 
of Title I schools is, as noted above, explicitly on the basis of a group, nontestscore 
variable; in this case, the proportion of students on AFDC as measured by the annual 
school district census. Our data consist in mean achievement scores on MAT in GE 
units for fall ("pretest") and spring ("posttest") for the five Title I and the five 
voucher elementary schools for each grade from second to fifth for the school year 
1972-1973, the first year of the voucher demonstration. 

Note, however, that for administrative reasons, the fall MAT testing in the 
voucher schools was delayed (it occurred in November 1972); hence, this is not in the 
strict sense "pretest," where the pretest score is obtained prior to the treatment (and 
should be unaffected by it; cf., e.g., Evans and Anastasio, 1968). Again, it is impossi- 
ble explicitly to allow for this event; we mention it simply as another point of caution 
when considering the results to follow. 

This, then, seems a reasonably realistic realization of the version of the Kenny 
(1974) model of selection on the basis of group characteristics. As a first cut, we will 
examine the relevant intercorrelations of the selection variable (proportion in the 
school on AFDC), the treatment variable, the fall (pretest) and spring (posttest) 
scores. These correlations appear in Table VIII-C.2. 

^ Note that this is a situation in which a great many researchers would reach up to their shelf ofLittle 
Jiffies and pull down the covariance model; some others, more sophisticated about these things, might 
elaborate the covariance model by correcting for unreliability in the pretest (cf., e.g., Cronbach and Furby, 
1970, the most likely source for many researchers of this bit of sophistication). This is meant to imply 
that Cronbach and Furby would recommend this model; they, among the most sophisticated of aJI 
analysts, precisely would oppose it. However, the notion of correcting for unreliability in such situations 
received probably widest circulation in their paper. However, given Kenny's model, it is easy to see that 
either of those models in this instance would yield biased estimates. 
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Table VIII-C.2 
CORREUTtONS FOR GROUP SELECTION MODEL 



Grade 
(1) 


r 

A.F 

(2) 


r 

(3) 


r 

(4) 


r 

F,S-A 

(5) 


r 
T,F 
(6) 


r 

T,F-A 
(7) 


r 
T.S 

(8) 


r 

T,S-A 
(9) 


r 
T,A 
(10) 


2 


-0,89-^ 


-0,82^ 


0.90^ 


0,64^ 


0.99^ 


0,94"^ 


0.90"^ 




-0.87-^ 


3 


2 

-0, 75 


-0,59^ 


0,75^ 


0,58^ 


0,75^ 


0,31 


0.53^ 


0.05 


-0,87-^ 


4 


-0,84^ 


2 

-0,79^ 


0,83^ 


0,48 


0.84^ 


0,40 


0,64^ 


-0.18 


-0,87-^ 


5 


'0,89-^ 


2 

-0,82 


0,94-^ 


0,81^ 


0,79^ 


0.05 


0.67^ 


-0.17 


3 

-0,87 



A 




proportion AFDC students. 




F 




fall (pre)test. 




C 




spring (post) test , 




T 




treatment (1 = voucher/0 = Title 


I). 


1 




0,01 - p < 0,05, 




2 




0.001 p -' 0,01, 




3 




p < 0,001, 




z 




r between x and y partialling z; 


r between residuals of 






jc and y after regressing each on 


z , 



An immediate notable feature of the data appears in the correlations of AFDC 
proportion and pretest in column (2) of Table VIII-C.2. It is clear thai, on an aggre- 
gate basis, selection of academic low-achievers by selecting schools with a high 
proportion of students on welfare works very nicely. Recalling that all of the correla- 
tions are based upon only N = 10 units (the ten schools), and that correlations of 
aggregate data (which do not take account of variability among individuals) are 
expectably higher than correlations of data on individuals, these correlations are all 
high (all p < .01) and negative (the higher the proportion on welfare, the lower the 
mean achievement score). The correlation of AFDC and treatment in column (10) 
(since these variables are available only at the school leveK the correlation is of 
course constant across grades) attest to the success of selection and to the degree of 
confounding of treatment and an almost certainly relevant variable in any causal 
function for achievement scores (Mosteller and Moynihan, 1972). The treatment-test 
correlations (columns (6) and (8)) are also predictably high.'* 

On the surface, the indica-ted comparison would seem to be, for each grade, the 
correlations in column (6) (treatment-pretest) with those in column (8) (treatment- 
posttest). But note that, while the latter are uniformly lower than the former, 
indicating a tendency over the year for the two groups to converge, they are all, 
especially the treatment-pretest correlation, high and positive (all p < .01). Also, we 
know definitely one of the reasons for the size of these correlations: the treatment 
was deliberately confounded with a potent influence on achievement; and knowing 

' Indeed, the treatment- pre test correlation for Grade 2 is almost unbelievably hish (the first entry in 
column <6): rp.F ^ f)-99). A quick look at Table V!II-C.l, however, will show what is going on. This is one 
of those very rare instances wherein the two wi thin-groups distributions are completely nonovcrlappinf^: 
this, together with the comparatively small within-group variance, especially in the Title I group, ac- 
counts for the near-perfect correlation. (Cf , e.g.. Haggard, 1958.) 

ERLC 
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this, we would be less than imaginative if we did not include this known injfluence 
in our model. The evidently wise procedure, then, would be to compare, not the 
zero-order treatment-test correlations, but the /?ar^/a/ correlations, controlling for 
the known selection variable. This will cost us one degree of freedom in a situation 
where df s are already very scarce; on the other hand, it is clearly dictated by realism 
and power considerations. 

The partials appear in columns (7) and (9) of Table VIII-C.2; as an eyeball 
comparison with columns (6) and (8), respectively, readily shows, the results are 
rather dramatic. Six of the eight partials, as opposed to all of the zero-orders, are 
not different from zero at any commonly acceptable level of significance (with df = 
7, P(r > ,58) - ,05); for the abnormally high second grade correlations, the decrease 
in one of the two is substantial,** Having introduced into our estimation procedure 
the selection variable, we may proceed to test the null hypothesis of no treatment 
effect in terms of the null that the treatment-test correlations are equal, ^ 

Notice that the two treatment-test correlations are themselves correlated, in the 
sense of a shared array (McNemar, 1969; Peters and Van Voorhis, 1940; Snedecor 
and Cochran, 1967). This means, of course, that the standard error of their difference 
must take account of this correlation, i,e., the standard error is almost certai i :ly less 
than the square root of the sum of the two variances. Formulae to adjust the 
standard expressions assuming independence were derived by Pearson and Filon 
(1898) and are given without derivation in Peters and Van Voorhis (1940), The 
results of these tests appear in Table VIII-C.3, Perhaps it is worth noting that, while, 
say 90 percent confidence limits about any of the correlations in the three higher 
grades would include zero, this will not sufl^ce as a test of the hypothesis that, within 
any grade, the two correlations estimate a common parameter. That is, it is possible 
for neither to be significantly different from zero individually, yet for the difference 
between them to be significantly nonzero. 

Although for purposes of this analysis, we could have introduced grade in school 
as an additional predictor, it seemed clearer to do a separate analysis at each grade 
level, although the analyses are clearly not independent. As a glance at Table 
V111-C.3 shows, however, the results are quite consistent. In no case save the aber- 

' Note that this partialling procedure is not an application of the covariance model. Analysis of 
covariance mandates wilhin groups regression of the response variable on the covariate; and an assump- 
tion ofthe model is that the slopes of these within-groups functions estimate a common parameter. Here, 
on the contrary, the slope estimate is based on the treatment SS, not the error SS or the "error line" 
(Cochran, 1957). For the treatment-test correlations are not within-groups functions, nor does the model 
indicate that they should be. Note also that the model assumption that the group selection factor is 
perfectly stable over time cannot be tested here; the AFDC census is taken only once each year, and this 
factor was applied to both treatment-test correlations. However, analysis included elsewhere in this 
report indicates that in fact even year-to-year shifts in this variable are not substantial, and it is unlikely 
that shifts of the year-to-year magnitude would occur within years, even given the mobility rate of about 
30 percent. 

Other possible control variables are, we should note, available. These are in the form of additional 
SKS indicators, such as proportions of ethnic minorities and so on. However, it is clear that we can ill 
afford to spare the degrees of freedom lost by further partialling. Moreover, from our perspective, it seems 
reasonable to regard these other SES indicators as just that: alternative indicators of a single-factor 
variable which we might call "poorness." This might suggest that we use scores from the first principal 
component of the indicator matrix; but, given that the AFDC variable is the selector (and that it is, in 
t hat sense, perfectly reliable L we might more reasonably use scores on a factor fixed through the AFDC 
point. This, however, would inevitably attenuate the factor (as compared with the principal component); 
and the attenuation would probably be greater than we would expect on the assumption that we had 
collected the data: these SES indicators are collected at different points in time, for different purposes, 
and apparently by different people, with no obvious effort or reason to assure comparability. Hence, it 
seems not only practical, but a somehow neater test of the model to proceed as above. 

We should mention that the AFDC control was chosen in advance of looking at the data, simply on 
the basis of knowledge of the selection process; the estimate, then, should not unduly capitalize on chance, 
whence lead to abnormal shrinkage in on a replication. (Lord and Novick, 1968; Nunnally, 1967.) 



Table VIII-C,3 



UIFFERKNCKS BlvTWEEN CORRELATIONS OF TREAT^tENT AND 
PRETEST AND POSTTEST CORRJiCTEU FOR GROUP 
SELECTION VARIABLE (AFDC) , BY GRADE 









* 




Grade 


''t.S'A ^T,F'A 


^2) 


t 


P(2-tail) 


(1) 


(2) 


(3) 


(4) 


(5) 


2 


0,28 


0,18 


1,56 


0,20 • p • 0,10 


3 


0,26 


0,29 


0,90 


• 0,20 


4 


0,58 


0,41 


1,41 


• 0,20 


5 


0,22 


0,20 


1,10 


• 0,20 



*DF = 7, 



rant second grade is any value of /greater than about 1.4 (p > .20); for the second 
grade, t = 1.56 (df = 7, .20 > p > .10). 

In short, the results of the analysis would support the conclusion that we cannot 
reject the null hypothesis of no treatment effect at any generally acceptable level 
of significance. 

Given this result, of course, it should be borne out by an analysis of the depend- 
ent variable, the test scores themselves. The dependent variable in this model will 
be the difference between standardized scores for spring and fall; and, in view of the 
introduction of the selection variable as a factor in the equation, the ultimate 
dependent variable will be the standardized residuals from the between-groups 
regression of test score on AFDC. The results of these tests appears in Table VIII-C.4. 
It is well to note that this test, for each grade in turn, is exactly equivalent to a test 
of the Treatment x Time interaction for a repeated measures analysis of variance 
(Cochran, 1972; Green and Tukey, 1960; McNemar, 1969; Winer, 1971). That is, had 
we formally structured this as an ANOVA model of the repeated measure sort, the 
Treatment x Time interaction is just a test of the difference between the Treatment 
groups of the differences between pre- and posttest, which would be the test of 
interest. 

Since the largest value of t is less than 1.00, it is apparent that the results are 
parallel to those from the previous test; indeed, this test is, demonstration purposes 
apart, superfluous. 

* * * 

In the end, we find ourselves asking the imperative question of any analysis, 
have we milked the data dry? In a sense, of course, we have not; a good deal of 
dredging remains to be done. But, in answer to our principal question: Is there, 
granted all ot^our caveats about the situation and the data — is there any apparent 
reason in the data as given to reject the null hypothesis that there was no significant 
difference between these two educational treatments? — our three analyses seem to 
converge: no difference of statistical or practical significance is visible. 

Granted, then, that these data are hardly ideal, either in level of aggregation, 
congruence of conditions of testing, extent in time, and so on; granted all this, what 
do they seem to tell us about the influence of the voucher demonstration as opposed 
to the Title I programs? If we take the Title I programs as a base, given that we can 
control explicitly on the selection factor, each of our analyses indicates that the 
voucher demonstration had little or no effect, positive or negative, on academic 
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Table VIII-C.4 



TESTS OF TREATMENT DIFFERENCES IN GROUV 
SELECTION MODEL 



Grade 


T - V 




* 

t 


P 


2 

ri 


2 


-0.246 


0.527 


-0.47 


• 0.50 


0.027 


3 




0.577 


-0.40 


> 0.50 


0.019 


4 


-0.506 


0.617 


-0.82 


> 0.40 


0.078 


5 


-0.190 


0.386 


-0.49 


> 0.50 


0.029 



NOTE: T = mean of Title I, V = mean of 
voucher. 



*DF - 7. 



achievement as measured here. These results are prima facie incongruent with the 
findings reported in Appendix VIII-B, on the basis of a different set of reading tests, 
mandated by the State of CaHfornia. Analysis of 1973-1974 data, which include both 
MAT scores and state-mandated test scores for all voucher and nonvoucher schools 
in the district, should help to resolve these differences. 

If we believe that the confusion attendant upon putting a massively reorganized 
educational program into the field, and the shorter pre-post interval for the voucher 
schools, would tend to affect test scores adversely, then we have even stronger 
reasons to conclude that, whatever the effect (if any) of the demonstration, it did not 
harm the students sufficiently in the aggregate to depress their scores unduly. 
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Appendix VIII-D 
AFFECTIVE TESTING 

by Theodore S. Donaldson 



INTRODUCTION 

Interest in the improvement and measurement of affective states (motivation, 
attitudes, seil esteem, self awareness, happiness, and other personality variables) 
has steadily increased in recent years, motivated in part by the lack of success in 
modifying cognitive achievement through "standard" educational innovations. The 
importance of affective grcvvth is often defended on the basis of two arguments. One 
view contends that affective factors are important because they are believed to be 
the major determinant of cognitive achievement, and the other view holds that 
growth in affective rather than cognitive factors is the more relevant goal of educa- 
tion. These views are certainly not mutually exclusive, and in fact, the distinction 
between affect and cognition is extremely artificial: attitudes and motivation have 
strong intrinsic cognitive components, and cognition involves affective components. 
Piaget and Inhelder (1969), among others, have made a strong argument for consid- 
ering affect and cognition as ^^inseparable and irreducible" (p. 158). Recognizing the 
artificiality in an affective-cognitive dichotomy, we proceed with arguments for 
including measures of what are called affective factors. 

How educational programs might influence affective growth is a matter of much 
speculation; however, it seems that the expert consensus is that the primary factors 
in the formation of affective states are in the child's home, and mostly in the early 
years. A lengthy study by Coopersmith (1967) places the antecedents of self esteem 
in the home, and Gordon (197(3) briefly reviews some of the evidence supporting the 
relevance of home factors. Very little can be found supporting the notion that 
affective states can be seriously modified by early school influences. The school can, 
however, support existing affective predispositions. For example, children with low 
self esteem may rarely find rewarding and successful experiences in school and their 
low self esteem is thus supported. 

At least part of the reason for the school's lack of success in modifying affective 
growth is the lack of clearly defined processes that could bring about growth, in spite 
of often stated objectives to do so. Merely stating that an objective is improving 
student self esteem says nothing about how it is to be done, and if schools are to bring 
about affective change, it must be through careful definition of processes which are 
thought to be effective. It is generally agreed that foremost among these processes 
is the construction of successful experiences for the early learner, although it is not 
exactly clear how this can be accomplished, especially in view of the heavy influence 
ofnonschool factors on affective growth and the limitations imposed by the schools' 
commitment to graded learning. 

In any event, to effectively evaluate the influence of an educational program on 
affective growth would require a careful investigation of the proposed and imple- 
mented classroom processes, and the definition of measures which bear on those 
processes. In this case, comparisons across programs may not be meaningful, unless 
programs have nearly identical objectives and aspire to the same processes. Regard- 
less of specific objectives, however, it is reasonable to assess the relative influence 
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of various affective factors on achievement and to make these comparisons across 
programs. Further, there are a number of affective factors that seem relevant to all 
educational programs regardless of stated objectives. Finally, there is a growing and 
convincing body of evidence showing that affect and cognition are empirically as 
well as theoretically related — positive affective states are associated with high 
achievement. Thus, if educational programs do improve affect, achievement should 
also improve. There is of course some question as to what kind of achievement is 
measured, and evaluations typically focus on only a narrow range of possible cogni- 
tive outcomes. 

Even though affect and achievement are correlated, the implementation of pro- 
grams which aim at affective growth could introduce problems for evaluation in 
comparing achievement among various types of programs. The graded, fixed cur- 
riculum approach to education (in which all first graders are exposed to first grade 
material, etc.) does not allow for some approaches to affective development. For 
example; a school which focuses on, say, the development of self esteem through play 
activities, ma> show very poor results on reading tests, at least in the early grades; 
and those programs which do not aspire to a graded curriculum cannot be evaluated 
by comparison with programs that do so, at all grade levels. In this case one needs 
to evaluate relatively long-term effects, perhaps looking at achievement scores at 
the exit of elementary school. 

At the present time there is little information on specific voucher educational 
processes or even what the affective objectives of various programs are. Moreover, 
even given affective objectives, there is no assurance that the proposed processes are 
in fact implemented. We would go so far as to say that it is probably impossible for 
many teachers to implement many affective growth (or other) strategies. Lacking 
specific information on processes, the approach for the affective component of the 
evaluation is to attempt to measure several factors which are believed to be general- 
ly important, and which are often stated as objectives in innovative programs. 

Probably the most widely researched affective component is that of self concept 
or self esteem, although, as pointed out by Zierkel (1971), there is a proliferation of 
definitions and measures, and this probably accounts for some of the confusion and 
inconsistencies in reported results. For example, the relationship between self con- 
cept and SES is extremely clouded and many contradictory results can be found. In 
spite of this, self concept measures are widely used, and low self concept is often 
associated with learning problems, especially among Mexican-American children 
(Hernandez, 1973, Leonetti, 1973). Everything considered, measures of self concept 
appear to be of fundamental importance in the affective battery. 

Another affective factor which has received considerable attention is that of 
students' attitude toward school, and in some cases (especially older children), their 
expectations and aspirations. One would expect that attitudes toward school would 
be related to self concept and achievement, although the possibility exists for pro- 
gram effects on one of these factors but not the other. Analysis of changes in the 
(possibly) complex relationships between self concept, attitude toward school, and 
programs should prove interesting. 

A number of other affective factors appear interesting and important; however, 
constraints imposed by the necessary length of the battery (dependent on the time 
available for testing) required that only the most promising tests be included in the 
first year's testing. Measures of anxiety, motivation, locus of control, alienation, and 
other factors could prove productive and some may be added later if the present 
battery can be shortened, or if some current measures turn out to be unimportant. 

Analyses will eventually focus on the relationships between affective data and 
achievement, parents' attitudes, SES factors, and, of course, educational program 
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type. Specilically, the analyses will attempt to determine the relative importance of 
various factors on achievement, and most importantly, whether or not some pro- 
grams are more efiective than others given certain student characteristics. In this 
case, "characteristic" is defined in the broadest sense, including in addition to the 
allective lest profiles such factors as SES and parent qualities. We will also of course 
determine changes in affect if they occur and attempt to determine the factors that 
appear responsible. 



CHOICE OF TEST BATTERY 

In general, we define two purposes for the affective test battery. The first is to 
measure affective change over time, and the second is to use affective measures to 
identify some student characteristics. These are, of course, not necessarily independ- 
ent, and are very likely to be confounded over time, i.e., affective growth may change 
the characteristic. Although we cann^. rule out the possibility of affective growth 
and its determination by the kinds of tests used in the demonstration, we do believe 
that short-term changes in affect probably do not occur for reasons discussed above. 
There is some empirical evidence to support this.* 

The primary purpose of the affective battery is to identify specific student char- 
acteristics that differentiate success in different types of programs. This is generally 
referred to as aptitude-treatment interaction, Treatment in this case refers to an 
educational program, or more accurately, to specific program features (such as 
structure). 

Although studies of aptitude-treatment interaction are becoming more preva- 
lent,"' there is still little conclusive evidence supporting any particular interaction, 
although much of the l:^.ck of success in isolating the effect is probably attributable 
to the primitive state of the art of affective measurement. For this reason the current 
affective measurement program must be considered in large part exploratory. 

In our view, the basic criterion for a good affective test is one that differentiates 
success in the various education programs in terms of student characterstics where 
student success is defined by some school outcome measure. Matters of test reliabili- 
ty and the like are important, but these statistical properties of tests generally gain 
fa** more attention than does their ability to discriminate among students in terms 
of some criterion measure (other than the test score itself). Student success is differ- 
entially indicated by program when: 

1. Test profiles (subtest scores within or between tests or both) for success or failure 
are substantively different among programs, or 

2. Regression of affective scores against success produces slopes for the same pro- 
files that are different among programs. 

Both require high correlations between some measure of program success and 
affective scores. 

In the first year of the voucher demonstration, several tests for each affective 
factor discussed above were administered. The purpose of the first year's aflTective 

' Personal communication with Coopersmith also supports this. See also S. B. Khan, "Aff'ective 
Correlates of Academic Achievement: A Longitudinal Study,*' Measurement and Evaluation in Guidance. 
Vol. 3. 1970. pp. 76-«0. 

^ L. J. Cronbach and R. E. Snow, Final Report: Individual Differences in l.earninf^ Ability as a Function 
of Instructional Variables. Stanford University Press, Stanford, Calif., 1969. Also see G. H. Bracht, 
"Experimental F'actors Related to Aptitude-Treatment Interactions/' Review of Educational Research. 
Vol. 40. 1970. pp. 627-646. 
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test program was to determine the usefulness (in terms of the criteria defined above) 
of various affective tests for future use in evaluating the voucher demonstration. It 
is important to remember that this first evaluation of the test piogram does not 
attempt to analyze programs, schools, or other factors — the only purpose is to select 
tests for future evaluations. 

The affective test battery consisted of five tests administered at the request of 
and scored and analyzed by The Rand Corporation, and two that v ^re administered, 
scored, and analyzed by the Alum Rock School District. The five tests that Rand 
administered are: The Coopersmith Self Esteem Inventory (Coopersmith, 1967), the 
Sears Self Concept Inventory (Sears, 1964), the Purdue Social Attitude Scale, the 
Primary Self Concept Scale (Leonetti, 1973), and the Primary Mental Abilities Test 
(PMA). The first two were given to a small sample of students in kindergarten 
through 6th grade. Rand was unable to analyze the districts' two tests because 
students usually did not put their ID numbers on these test papers, and it was not 
possible to match scores to students. These two tests, originally developed by the 
Cincinnati school district, were surveys of student attitudes about self and school — 
one for the upper elementary grades and one for the lower. 

The sample structure for each affective test turned out to be more complex than 
originally planned, and this had direct bearing on the kinds of analysis used on the 
data. This resulted primarily from the fact that teachers were allowed to not give 
affective tests if they so desired. Since the purpose of the first year's affective testing 
was to select one of two tests at each grade level, the complications and loss of 
information introduced by the sample structure were not serious, except that the 
analysis is more difficult to follow. 

In very brief summary, the analysis indicated that the Coopersmith test best 
suits our purposes in the upper elementary (4-8) grades, and the Purdue test was 
slightly preferable for the lower grades. The results of the Coopersmith data indicate 
strong possibilities for the presence of an interaction between program (type) and 
student characteristic. A number of other results appeared interesting, but in view 
of the confounding between variables introduced by the sample structure, it is 
difficult (and probably unwise) to discuss them. While showing interesting results 
(very strong interactions of subscale score by program type), the PMA will not be 
used in the immediate future, primarily because it is too time consuming. 
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